QA / Automation Engineer Agentic AI

We are looking for a QA / Automation Engineer

This role involves building and maintaining automated tests for an agentic, LLM-driven application, focusing on creating a new Python-based framework for evaluating agentic systems.

Responsibilities:

• Build and maintain automated tests for an agentic, LLM-driven application, extending beyond conventional UI test automation.
• Develop a new, Python-based framework for evaluating agentic systems, including LLM-as-judge style evaluators and state checks for conversational/agentic flows.
• Utilize Playwright for front-end automation where applicable, while focusing on the Python-based evaluation layer for AI-driven behavior.
• Collaborate closely with developers in a pod structure, as QA is distributed and collaborative rather than siloed.

Requirements:

• Comfort operating with a light, UAT-style formal QA process, where quality ownership is shared with developers.
• Ability to work in an undefined, evolving testing landscape as the framework is being built and expanded.

Required Skills:

• Proficiency in Python, as it is the primary language for the role's evaluation/automation framework.
• Experience or strong aptitude for testing AI/LLM-based systems, including evaluators, LLM-as-judge techniques, and state/behavior validation for agentic or conversational systems.
• Working knowledge of Playwright for the front-end automation layer, though it is not the core skill being hired for.

Preferred Skills:

• Industry domain knowledge in HCM.