Role Summary
\n
We are seeking a Senior Test Automation Engineer & Quality Engineering Lead to establish and scale quality engineering for a modern AI platform and AI-enabled applications.
\n
\n
This is a hands-on, player-coach role responsible for defining the automation strategy, building reusable frameworks and critical test suites, integrating quality into CI/CD, and helping engineering teams increasingly own product quality.
\n
\n
The role spans web and mobile applications, APIs and backend services, data and integrations, and AI/agent behavior.
\n
\n
Key Responsibilities
\n
Test Automation & Continuous Quality
\n
- \n
- Define an automation-first strategy across UI, APIs, services, integrations, data, and AI workflows.
- \n
- Build reusable automation frameworks and high-value regression suites, with API/service-level testing as the foundation.
- \n
- Automate critical web, mobile, and end-to-end journeys, including authentication, authorization, multi-tenant behavior, asynchronous interactions, and failure scenarios.
- \n
- Integrate automated tests and quality gates into CI/CD pipelines.
- \n
- Establish test execution monitoring, failure triage, reporting, and flaky-test management.
- \n
- Automate critical performance, latency, resilience, concurrency, and dependency-failure scenarios.
- \n
\n
AI, Agent & Data Quality
\n
- \n
- Establish repeatable evaluation and regression testing for Generative AI and agentic applications.
- \n
- Validate appropriate tool and data usage, workflow and authorization boundaries, human-approval controls, and handling of missing or conflicting information.
- \n
- Validate groundedness, factual accuracy, relevance, evidence quality, hallucination risk, instruction adherence, latency, and reliability.
- \n
- Test retrieval, tool-calling, multi-step agent workflows, and AI application failure scenarios.
- \n
- Automate data-quality validation covering freshness, integrity, completeness, transformation, traceability, and tenant isolation.
- \n
- Establish representative synthetic and simulated datasets for deterministic regression and AI evaluation.
- \n
\n
Engineering Leadership
\n
- \n
- Build the initial automation architecture and highest-value test suites.
- \n
- Lead automation design, code quality, and engineering standards.
- \n
- Partner with software, platform, data, AI, and DevOps engineers to embed testability early.
- \n
- Participate in architecture and design reviews and mentor engineers contributing to shared automation frameworks.
- \n
- Promote the right mix of unit, API, integration, end-to-end, performance, exploratory, and AI evaluation testing while driving distributed quality ownership.
- \n
\n
Experience & Technology Skills
\n
Strong candidates will have:
\n
- \n
- Senior-level test automation or quality engineering experience, including designing automation frameworks from the ground up.
- \n
- Strong Python skills for API, service, data, and AI test automation.
- \n
- Experience with TypeScript/JavaScript and modern web application automation; TypeScript preferred.
- \n
- Strong web, API/service, integration, and end-to-end automation experience.
- \n
- Experience testing React-based applications; Playwright or comparable browser automation preferred. Ant Design experience is desirable.
- \n
- Experience with automated testing of iOS and Android applications; cross-platform mobile automation is desirable.
- \n
- Strong experience testing REST APIs, asynchronous services, authentication, authorization, and multi-tenant systems.
- \n
- Strong SQL and data-validation skills across relational, operational, time-series, and document-oriented data.
- \n
- Hands-on experience testing applications and services deployed on AWS; AWS application, data, container, and observability services are strongly preferred.
- \n
- Experience testing applications built with OpenAI models/APIs or comparable production LLM platforms; OpenAI experience preferred.
- \n
- Hands-on experience with Generative AI and agentic systems, including retrieval, grounding, tool-calling, multi-step workflows, behavioral regression, hallucination/evidence validation, and AI observability.
- \n
- Experience integrating automated testing into Git-based CI/CD workflows; Bitbucket and Bitbucket Pipelines experience preferred.
- \n
- Experience with Docker/containerized applications and container-based test environments preferred.
- \n
- Strong debugging and failure-analysis skills using application logs, metrics, traces, APIs, data, and infrastructure.
- \n
- Candidates should be comfortable contributing production-quality automation code and working directly with software engineers.
- \n
\n
Success in the Role
\n
Success means establishing:
\n
- \n
- Reusable automation frameworks and quality standards.
- \n
- Strong API/service regression coverage with focused UI and end-to-end automation.
- \n
- Continuous data-quality and tenant-isolation validation.
- \n
- Repeatable AI/agent evaluation and behavioral regression testing.
- \n
- Actionable CI/CD quality signals and rapid failure diagnosis.
- \n
- Increasing quality ownership across engineering teams.
- \n
\n
\n
The goal is to provide fast, trustworthy quality signals that enable teams to release AI-enabled products confidently and efficiently.
\n