AI Engineer - Agent Development

CP Axtra PCL

NY

JOB DETAILS
SKILLS
Artificial Intelligence (AI), Caching, Computer Science, Computer Security, Cost Control, Data Collection, Error Handling, Injections, Instrumentation, Metrics, Microsoft Windows Azure, Production Systems, Python Programming/Scripting Language, Retail, Software Engineering, Technical Leadership, Test Design
LOCATION
NY
POSTED
4 days ago

Head Office

Other

Highlight

The AI Engineer builds production agents end-to-end on an AI-native retail decisioning platform - prompt design, tool definitions, multi-step workflows on the agent runtime (LangGraph, CrewAI, or chosen framework), evaluation harnesses (golden sets, regression gates, multi-step replay), human-in-the-loop gate integration, and per-agent cost optimisation. The role consumes platform-provided LLM and vector services; it does not rebuild that platform.

Remote candidates outside of Thailand are welcome to apply.

Responsibilities

  • Build agents on the platform's agent runtime - prompt design, tool definitions, multi-step workflows, error handling - and ship them with eval harness, human-in-the-loop gate config, observability instrumentation, cost meter, and runbook.

  • Co-design agent specs with Tech Lead Applications and Suite Product Owners; partner with ML Engineers on classical ML model integration into agents.

  • Author golden sets per agent - domain-specific test cases capturing must-pass behaviours; build regression gates in CI so no agent ships without eval-pass.

  • Implement multi-step conversation replay for agents with stateful interactions; use LLM-as-judge patterns where appropriate; instrument human feedback collection.

  • Configure HITL gates per agent and per agent plan; implement gate-progression evidence collection (Shadow data, accuracy metrics, override frequency).

  • Own per-agent cost meter - tokens, vector queries, model inference; report monthly; tune model routing and implement caching strategies where appropriate.

  • Consume the enterprise LLM Gateway via standard SDK; partner with platform AI engineering on embedding model selection and retrieval relevance tuning.

  • Mentor seed-programme engineers and contribute to the agent-engineering playbook.

Qualifications

  • Bachelor's or Master's degree in Computer Science, AI / ML, or a related discipline.

  • 5+ years software engineering with 2+ years shipping LLM-based or agentic systems to production.

  • Production agent or multi-step LLM workflow experience - LangGraph, CrewAI, AutoGen, DSPy, or custom.

  • Strong Python; comfortable with async, observability, testing.

  • Hands-on with at least one major LLM provider (Azure OpenAI, Anthropic, Bedrock, Vertex).

  • Eval-driven LLM development - golden sets, LLM-as-judge, regression gates, multi-step replay.

  • HITL gate / agent governance - has shipped agents with explicit gates, not autonomous-by-default.

  • Prompt injection / data leakage / PII handling - designs and tests defences.

  • Open-source contributions to agent frameworks (LangChain / LangGraph / DSPy).

  • Multi-agent system at scale in production; retail / commerce / fintech agentic workflows (supplier onboarding, contract intelligence, comparable).

  • Causal inference exposure (DoWhy / EconML); Thai-language NLP (PyThaiNLP, WangchanBERTa, SEA-LION, Typhoon).

  • Vendor certifications such as Databricks Generative AI Engineer or Azure AI Engineer Associate.

About the Company

C

CP Axtra PCL