Job Description
Overview:
TekWissen is a global workforce management provider headquartered in Ann Arbor, Michigan that offers strategic talent solutions to our clients worldwide. Our client is a leading global technology consulting and digital transformation organization that helps enterprises modernize their business operations through innovative cloud, data, and customer engagement solutions. With a strong focus on delivering scalable technology platforms and customer-centric digital experiences, the organization empowers businesses to drive growth, improve operational efficiency, and enhance customer relationships.
Job Title: QA Engineer for AI Products/Solutions
Location: Bellevue,WA 98008
Duration: 6 Months
Job Type: Temporary Assignment
Work Type: Onsite
Location: Bellevue,WA 98008
Duration: 6 Months
Job Type: Temporary Assignment
Work Type: Onsite
Job Description:
- Design and execute test plans for AI/ML-driven features, including model outputs, prompts, and integrated application behavior
- Build and maintain automated test suites covering functional, regression, integration, and API testing
- Evaluate model outputs for accuracy, consistency, bias, hallucination, and edge-case failures
- Develop evaluation frameworks and golden datasets/test cases to benchmark model performance over time
- Test prompt engineering changes, model version upgrades, and fine-tuning outputs for regressions
- Perform adversarial and red-team style testing to surface safety, security, and robustness issues
- Validate data pipelines feeding into AI models (data quality, schema, drift detection)
- Collaborate with data scientists/ML engineers to define acceptance criteria and quality metrics for models
- Test latency, scalability, and reliability of AI services under load
- Contribute to CI/CD pipelines, integrating automated and model-evaluation tests
- Hands-on experience testing LLM-based products (chatbots, copilots, RAG systems, AI agents) - designing test cases for non-deterministic, generative outputs
- Practical experience with AI/LLM evaluation frameworks (e.g., Ragas, DeepEval, LangSmith, Promptfoo, OpenAI Evals, TruLens) - building eval suites, scoring rubrics, and golden datasets
- Working knowledge of eval metrics for generative AI: hallucination rate, faithfulness/groundedness, relevance, answer correctness, toxicity/bias scoring, BLEU/ROUGE/semantic similarity where applicable
- Experience with prompt regression testing - validating prompt changes and model/version upgrades against baseline eval sets
- Strong proficiency in Python for writing eval scripts, test harnesses, and data validation logic
- Familiarity with SQL and data validation techniques
- The environment is primarily Microsoft Azure-based, with Snowflake also playing a key role.
- Their GenAI initiatives largely leverage OpenAI and Claude models.
TekWissen Group is an equal opportunity employer supporting workforce diversity.
Numbers & Facts
| Location | Bellevue, WA |
| Industry | Computer/IT Services |
| Salary | $57.91 |
| Company Size | 100 to 499 employees |
| Year Founded | 2009 |
| Website | http://www.tekwissen.com/ |
About Company
WE THE TEKWISSEN PEOPLE
TekWissen offers you a broader portfolio of services, industry-leading solutions, and the meaningful innovations that give you greater flexibility and speed to respond to market dynamics, reduced costs and risk to improve enterprise performance, and increased productivity to enable growth.
To keep pace with global market demands, TekWissen keeps its finger on the pulse of change. Our organized approach to guiding a project from its inception to closure. Managing projects is becoming more and more important as we enter the digital era. To cope with the pace that this transition demands, a method is required to manage projects so they can yield quality work, while incorporating efficient use of time and resources.
Project involves identifying which quality standards are relevant to the project and determining how to satisfy them.
It is important to perform quality planning during the Planning Process and should be done alongside the other project planning processes because changes in the quality will likely require changes in the other planning processes, or the desired product quality may require a detailed risk analysis of an identified problem. It is important to remember that quality should be planned, designed, then built in, not added on after the fact.
Capabilities and accomplishments in one TekWissen business enhance the opportunity for success in the others. Put simply, TekWissen's unique combination of attributes promotes success.Skills
- Application Programming Interface (API)unmatched
- Artificial Intelligence (AI)unmatched
- Artificial Intelligence (AI) Agentsunmatched
- Benchmark Testingunmatched
- Business Growthunmatched
- Business Operationsunmatched
- Cloud Computingunmatched
- Consultingunmatched
- Continuous Deployment/Deliveryunmatched
- Continuous Integrationunmatched
- Conversation Engineunmatched
- Customer Relationsunmatched
- Data Managementunmatched
- Data Modelingunmatched
- Data Qualityunmatched
- Data Scienceunmatched
- Data Setsunmatched
- Diversityunmatched
- Functional Testingunmatched
- Integration Testingunmatched
- Metricsunmatched
- Microsoft Windows Azureunmatched
- Operational Improvementunmatched
- Operational Strategyunmatched
- Performance Modelingunmatched
- Process Improvementunmatched
- Python Programming/Scripting Languageunmatched
- Quality Assuranceunmatched
- Quality Metricsunmatched
- Regression Testingunmatched
- SQL (Structured Query Language)unmatched
- Scalability Testingunmatched
- Technical Deliveryunmatched
- Test Automationunmatched
- Test Caseunmatched
- Test Designunmatched
- Test Harnessunmatched
- Test Scriptsunmatched
- Test Suiteunmatched
- Testingunmatched
- Training Data Setsunmatched
- Workforce Managementunmatched
Level up your application
Professional resume templates
Browse dozens of recruiter approved resume templates, layouts and formats. Choose your favorite and make it your own in minutes.
Free resume templatesFree resume builder
Improve your existing resume or start from scratch and create a standout, ATS-friendly resume. Add job-specific content, download and apply.
Free resume builder