AI/ML QA Tester

Job Details

• Location: Rosemead, CA
• Duration: 6 months
• GBaMS ReqID: 10933405
• Role: Tester / QA
• Experience Required: 3–7+ years
• Domain: AI / Machine Learning / Quality Assurance

Role Overview

• Ensure AI products behave:
  • Accurately
  • Safely
  • Fairly
  • Reliably
• Validate AI products across real-world scenarios and use cases.
• Partner with:
  • Product teams
  • Engineering teams
  • Subject Matter Experts (SMEs)
• Validate model behavior, data quality, and trust risks throughout the AI lifecycle.

Key Responsibilities

• Test AI outputs for:
  • Accuracy
  • Consistency
  • Edge cases
  • Failure modes
• Validate training and inference data quality.
• Identify potential:
  • Bias
  • Data leakage
  • Data quality issues
• Define and execute model evaluation and regression tests.
• Reuse and expand the Golden Dataset for regression testing across:
  • Builds
  • Prompt changes
  • Model changes
  • Product releases
• Test AI behavior within end-to-end user workflows and integrations.
• Support Responsible AI testing, including:
  • Fairness
  • Explainability
  • Compliance
• Monitor model performance after release.
• Identify and track:
  • Model drift
  • User feedback
  • Regressions
  • Performance degradation
• Evaluate and improve test automation coverage.
• Communicate testing risks and findings clearly to stakeholders.
• Track and report quality metrics, including:
  • Accuracy
  • Precision
  • Recall, where applicable
• Clearly identify:
  • Missing content
  • Incorrect content
  • Hallucinated content

Required Skills

• Strong experience in:
  • QA test planning
  • Exploratory testing
  • User Acceptance Testing (UAT)
• Understanding of AI/ML fundamentals.
• Knowledge of AI/ML evaluation metrics.
• Strong data analysis skills.
• Ability to analyze and reason about model behavior.
• Ability to test Generative AI prompts and outputs preferred.
• Strong domain knowledge and user-centric mindset.
• Excellent documentation and communication skills.

Qualifications

• 3–7+ years of experience in:
  • QA
  • Software testing
  • Quality assurance/quality roles
• Experience working with AI- or data-driven products preferred.
• ISTQB certification is a plus.
• AI/Data-related training or certifications are a plus.
• Responsible AI training is a plus.

Key Skills / Keywords

• AI QA
• AI Testing
• GenAI Testing
• Machine Learning Testing
• Model Evaluation
• Regression Testing
• Exploratory Testing
• UAT
• Data Quality
• Golden Dataset
• Responsible AI
• AI Bias Testing
• Explainability
• Model Drift
• Prompt Testing
• AI Hallucination Testing
• Test Automation
• Accuracy / Precision / Recall
• Data Analysis