We are seeking a Senior Expert, Quality Engineer (QE) with deep expertise in quality engineering for AI-powered systems. In this role, you will own and lead end-to-end quality for your project while playing a key role in testing and evaluating complex, non-deterministic AI systems. You will help shape the quality strategy for the project and raise the overall quality bar for the teams you work with.
What You'll Do
Core Quality Engineering
Define quality strategies and own test planning from requirements through release , partnering closely with cross-functional teams
Design and execute scalable functional, regression, and integration testing strategies
Apply hands-on automation framework expertise across UI, API, and backend layers to support major product initiatives
AI & LLM System Testing
Test AI-driven functionality , including LLM agents, multi-agent workflows, and RAG pipelines
Validate AI outputs for accuracy, relevance, consistency, and edge cases across non-deterministic systems
Own evaluation frameworks (e.g., RAGAS) , test datasets, and regression baselines for AI features
Stay current with the evolving landscape of LLM evaluation, agent testing, and AI observability , and translate emerging techniques into practical testing approaches
Technical Leadership & Delivery
Lead QE team for your project by setting technical direction, defining quality strategy, and ensuring delivery confidence
Contribute to QE standards and frameworks within your project scope, establishing practices that other engineers can adopt
Integrate automated tests into CI/CD pipelines and partner with engineering leads to embed quality into the delivery process
Leverage data, risk signals, and production insights to continuously improve test coverage and effectiveness
What You'll Bring
8+ years of experience in Quality Engineering or Quality Assurance
Proven hands-on experience testing AI/LLM-based systems , including agents, multi-agent workflows, and RAG pipelines
Experience with AI evaluation approaches (e.g., RAGAS, LLM-as-judge, regression baselines) and non-deterministic testing strategies
Strong automation skills using Playwright or a comparable framework
Proficiency in JavaScript/TypeScript and/or Java
Solid experience with API testing and data validation
Demonstrated track record of leading quality end-to-end for complex projects , from strategy through execution
Ability to define and drive QE technical direction within a project or product area
Experience working in Agile delivery environments and contributing to fast-moving release cycles
Nice to Have
Hands-on experience with prompting, embeddings, or vector databases
Cloud experience (AWS preferred)
Exposure to AI-assisted development or testing tools (e.g., Cursor, Copilot)
Experience with AI observability or monitoring tools for production LLM systems
This is a hybrid role based in Hoboken, NJ. The ideal candidate should be able to work from the Hoboken office at least 2-3 days per week.
Applications will be accepted through July 7. This window may be extended depending on business needs.
Compensation at Pearson is influenced by a wide array of factors including but not limited to skill set, level of experience, and specific location. As required by the California, Colorado, Hawaii, Illinois, Maryland, Minnesota, New Jersey, New York State, New York City, Vermont, Washington State, and Washington DC laws, the pay range for this position is as follows:
The full-time salary range for this position is between $130,000 - $150,000
This position is eligible to participate in an annual incentive program, and information on benefits offered is here .
Who we are:
At Pearson, our purpose is simple: to help people realize the life they imagine through learning. We believe that every learning opportunity is a chance for a personal breakthrough. We are the world's lifelong learning company. For us, learning isn't just what we do. It's who we are. To learn more: We are Pearson.
Pearson is an Equal Opportunity Employer and a member of E-Verify. Employment decisions are based on qualifications, merit and business need. Qualified applicants will receive consideration for employment without regard to race, ethnicity, color, religion, sex, sexual orientation, gender identity, gender expression, age, national origin, protected veteran status, disability status or any other group protected by law. We actively seek qualified candidates who are protected veterans and individuals with disabilities as defined under VEVRAA and Section 503 of the Rehabilitation Act.
If you are an individual with a disability and are unable or limited in your ability to use or access our career site as a result of your disability, you may request reasonable accommodations by emailing TalentExperienceGlobalTeam@grp.pearson.com.
Job: Engineering
Job Family: TECHNOLOGY
Organization: Higher Education
Schedule: FULL_TIME
Workplace Type: Hybrid
Req ID: 24814