Sign upLog in
IT Quality Assurance Jobs

AI Quality Engineer

Apolis

Charlotte, NC

Apply
JOB DETAILS
SALARY
$55–$58 Per Hour
SKILLS
Analysis Skills, Application Programming Interface (API), Artificial Intelligence (AI), Benchmarking, Bug Tracking/Defect Management, Business Processes, Communication Skills, Computer Science, Continuous Deployment/Delivery, Continuous Improvement, Continuous Integration, Corrective Action, Cross-Functional, Customer/Consumer Behavior, Data Quality, Data Science, Data Sets, Detail Oriented, Documentation, Financial Services, Functional Testing, Information Technology & Information Systems, Integration Testing, Machine Tool, Management Consulting, Metrics, Process Improvement, Production Control, Production Systems, Python Programming/Scripting Language, Quality Engineering, Quality Management, Quality Metrics, Quality Monitoring, Regression Testing, Risk Modeling, Root Cause Analysis, SQL (Structured Query Language), Software Testing, System Integration (SI), System Validation, Team Building, Test Automation, Test Case, Test Data, Test Design, Test Plan/Schedule, Test Scripts, Test Strategy, Test Tools, Testing, Usability Engineering, Use Cases, User Interface/Experience (UI/UX), Validation Testing, Wealth Management
LOCATION
Charlotte, NC
POSTED
10 days ago
Role: AI Quality Engineer
Location: This is a HYBRID position requiring 2 in-office days at local Deloitte office per week. Travel costs are not reimbursable.
Duration: 6+ Months
 
Only W2.


"We are seeking an AI Quality Engineer to join an innovative and high-impact team building AI capabilities within enterprise systems for a leading wealth management client. In this role, you will define, execute, and continuously improve quality engineering practices for AI-powered applications, including LLM-based assistants, retrieval solutions, and intelligent workflows. You will help ensure these solutions are reliable, accurate, secure, and fit for enterprise use by building robust testing strategies across model behavior, system integrations, user experience, and operational performance. This is an opportunity to shape the next generation of enterprise AI in a highly visible environment, where innovation, rigor, and business value all matter. The ideal candidate brings a quality-first mindset, strong testing discipline, and practical experience validating complex systems in regulated environments.

Key Responsibilities

Design and execute test strategies for AI applications, including functional, integration, regression, usability, and non-functional testing
Validate LLM-based solutions for accuracy, relevance, consistency, safety, latency, and task completion across realistic business scenarios
Create test cases, evaluation datasets, and quality benchmarks for prompts, retrieval pipelines, orchestration flows, and end-user interactions
Partner with AI engineers, product owners, and business stakeholders to define acceptance criteria and measurable quality standards
Identify model failure modes, edge cases, hallucination risks, and workflow breakdowns, and recommend corrective actions
Support automation of AI testing, monitoring, and quality reporting across development and production environments
Test data flows, APIs, integrations, and system dependencies that support AI-enabled business processes
Contribute to governance and release readiness by documenting defects, risks, test results, and quality decisions
Monitor production feedback, user behavior, and operational metrics to help drive continuous improvement
Help establish repeatable AI quality engineering practices, tooling, and controls for enterprise delivery

Required Qualifications

Bachelor’s degree in Computer Science, Engineering, Information Systems, Data Science, or related field
6+ years of experience in quality engineering, software testing, QA automation, or related technical roles
Experience testing enterprise applications, APIs, data pipelines, or workflow-based systems
Experience evaluating AI or ML-based systems, including prompt behavior, output quality, or model-driven workflows
Strong understanding of test strategy, defect management, root cause analysis, and quality metrics
Familiarity with automation frameworks, test management tools, and CI/CD-aligned quality processes
Ability to work across technical and business teams to define and validate expected outcomes

Preferred Qualifications

Experience with LLM evaluation, prompt testing, RAG validation, or AI observability tools
Familiarity with Python, SQL, APIs, or scripting for test automation and data validation
Experience building synthetic datasets, golden test sets, or evaluation frameworks for AI use cases
Exposure to responsible AI, model risk, privacy, or governance controls in enterprise settings
Experience in financial services, wealth management, consulting, or other regulated environments

Core Competencies

Analytical rigor and attention to detail
Structured testing and defect analysis
Curiosity about model behavior and failure modes
Cross-functional collaboration
Clear communication of quality risks and trade-offs
Focus on reliability, user trust, and measurable outcomes"
 
 

About the Company

A

Apolis

Since 1996, RJT has provided successful SAP, Oracle, and IT consulting solutions and staffing services to clients around the world. The new Apolis brings you the same personalized service fortified with a greater array of IT solutions, global expertise, and cost-management strategies.

We are a global IT consultancy that seamlessly integrates experts and leading-edge solutions into your organization so you can focus on what really matters.

COMPANY SIZE
500 to 999 employees
INDUSTRY
Computer/IT Services
EMPLOYEE BENEFITS
Paid Sick Days, Employee Referral Program, Employee Events, Retirement / Pension Plans
WEBSITE
https://www.apolisrises.com/

Resume Resources

Free Resume TemplatesFree Resume Builder

Similar Job Searches

Remote IT JobsEntry Level IT Jobs