AI Evaluation Engineer

Role Descriptions:
Technical/Functional Skills 10+ years of experience building large-scale distributed systems + strong experience with LLM systems| agentic workflows or advanced ML infrastructureAI engineers with recent NodeJS/Javascript/Typescript experienceProven ownership of complex| cross-cutting agentic systems spanning multiple teams or products.Strong engineering fundamentals across backend systems| APIs| data pipelines| and cloud infrastructure.Deep experience across the agentic AI stack| including planning| tool use| memory| and evaluation.Fluency with AI-assisted and agentic development workflows.Comfort operating in ambiguous problem spaces and translating them into shipped| reliable autonomous systems.Ability to influence technical direction and align teams without formal authority.Experience in workflow engines| async processing| queues| and streaming systems.Languages: NodeJS/Javascript/Typescript|Python| GoAPIs and services: REST| gRPCCloud and infrastructure: AWS and/or GCP| KubernetesDistributed systems: event-driven architectures| including KafkaOrchestration Frameworks: LangGraph| LangChain| AirFlow| etcIntegration of commercial and open-source LLMs into agentic workflowsAgent and orchestration frameworks such as LangChain| LlamaIndex| Semantic Kernel| or CrewAI| with strong judgment about when to use frameworks versus building lighter-weight primitivesModel-level work using PyTorch and the Hugging Face ecosystem (embeddings| fine-tuning| inference tooling)| with some exposure to TensorFlowStrong schema| validation| and state management practices using tools such as Pydantic (Python) and Zod (TypeScript) Experience building agentic systems in fintech or other regulated industries.Experience as a founding engineer or early technical leader in AI-driven products.Demonstrated success delivering technically complex autonomous systems that customers actively rely on.Meaningful contributions to open-source AI or agentic frameworks.Familiarity with fine-tuning| model optimization and inference pipelines is a plus.Roles & ResponsibilitiesDrive technical direction for agentic AI initiatives| influencing architecture patterns| autonomy boundaries| and system design.Design| build| and operate production-grade agentic AI systems used across multiple products.Own and evolve shared agentic AI capabilities| including:Agent frameworks and orchestration layersPlanning| tool use| and memory strategiesRetrieval and grounding (RAG) pipelinesLLM infrastructure| inference| and model gatewaysEvaluation| observability| and safety tooling for autonomous systemsLead technical design reviews and help teams navigate tradeoffs involving autonomy| safety| reliability| scalability| and cost.Partner across teams to deliver complex| cross-cutting agentic AI initiatives from concept to production.Evaluate emerging models| techniques| and agentic patterns and translate the