AI/ML QA Engineer

The AI/ML QA Engineer will work in client team to deliver the high-performance AI/ML models. This role is heavily involved in quality assurance for AI and machine learning algorithms, ensuring they meet the defined accuracy, reliability and effectiveness within software applications. The ideal candidate will have proven experience in AI/ML model testing, a keen attention to details, and an unwavering commitment to quality control. This candidate must be comfortable working in a highly collaborative, fast-paced environment to coordinate with the developers in data science. This is a hands-on role, and we are looking for a candidate who has the technical acumen for quality assurance and a passion for rigorous testing and validation.

Job Responsibilities:

• Develop comprehensive test plans and create test cases to systematically evaluate AI's performance.
• Implement testing at various stages of model development and validate the model against business or technical requirements.
• Assess AI-generated responses to ensure they meet predefined quality standards
• Collaborate with developers to understand the expected model behavior and identify the root cause of bugs or issues.
• Monitor bug solutions and verify fixes in the AI/ML systems.
• Maintain detailed records of all QA activities, including test logs and documentation.
• Stay updated with the latest advances in AI/ML technologies, best practices, and testing tools.

Job Requirements:

• Bachelor's or higher degree in Computer Science, Data Science, or a related discipline.
• 5+ years of relevant experience in software quality assurance with demonstrated success of firsthand, creative testing.
• Experience in developing testing strategies, including test plans, test cases, custom tools, and testing architectures in AI/ML products.
• Experience in testing predictive and classification model outputs, such as Linear Regression, Random Forest, Gradient Boosting in terms of model accuracy, bias, and data validation.
• Experience in validating Gen-AI LLM outputs from automated document review and RAG methods for more accurate, relevant, and context-aware responses.
• Experience with test automation tools, such as Pytest and TensorFlow Extended.
• Excellent skills in Python or other equivalent scripting languages for Testing Automation.
• Excellent skills in problem solving, critical thinking, and effective communication.
• Experience in Pharmaceutical Industry & Clinical Trials Data is a plus.