Job #: 163765-1
Job Title: Content QA Analyst
Rate: $42/hourly
Location: Remote (PST hours)
Duration: Through 12/31/2026 (Possibility of Extension)
We are seeking a detail-oriented and customer-focused Content QA Analyst to ensure the quality, reliability, and effectiveness of content experiences across Microsoft 365 products, including templates, prompt templates, AI-generated content, and emerging Copilot experiences.
This role sits at the intersection of content, user experience, and quality assurance. The successful candidate will validate content performance, identify defects, evaluate AI-generated outputs, and help improve customer-facing experiences across productivity applications and AI-powered solutions.
The ideal candidate combines strong testing expertise with an understanding of content quality, usability, functionality, accessibility, and customer experience.
Required Skills & Qualifications
Education
- Bachelor's degree in Computer Science, Engineering, Information Systems, Content Automation, or a related field.
Experience
- Experience in quality assurance, content testing, quality control, usability testing, or related disciplines.
- Experience creating test plans, test cases, validation documentation, and test reports.
- Experience evaluating digital content and customer-facing experiences.
- Experience applying structured testing methodologies and quality assurance processes.
Core Skills
- Knowledge of Microsoft 365 applications and productivity workflows.
- Experience scripting with Python, PowerShell, or similar technologies.
- Strong understanding of QA methodologies, defect tracking, and quality frameworks.
- Analytical problem-solving and data interpretation skills.
- Excellent written and verbal communication skills.
- Strong attention to detail and commitment to quality.
Preferred Qualifications
- Experience evaluating AI-generated content or generative AI experiences.
- Familiarity with Copilot and AI-assisted productivity tools.
- Understanding of content quality standards, user experience principles, and design best practices.
- Experience working with automation frameworks.
- Knowledge of human evaluation methodologies and quality scoring rubrics.
Candidate Requirements
The ideal candidate will demonstrate:
- Hands-on experience in content testing, QA, or QC environments.
- Proven ability to create test plans, execute test cases, and document results.
- Experience communicating testing outcomes and recommendations to stakeholders.
- Familiarity with scripting languages such as Python, PowerShell, or similar technologies.
- Ability to work independently in a remote environment and manage multiple priorities.
- Customer-focused approach with a commitment to continuous improvement and quality.
Assessment Process
Candidates may be assessed through a combination of:
Technical Assessment
- Review of QA, content testing, and quality assurance experience.
- Validation of test planning, test execution, and defect management capabilities.
- Discussion of testing methodologies, quality processes, and reporting practices.
- Evaluation of scripting and automation knowledge.
Scenario-Based Evaluation
- Review of content quality evaluation techniques.
- Assessment of AI-generated content analysis and recommendation development.
- Defect identification and root-cause analysis exercises.
- Sample test planning and reporting activities.
Communication Assessment
- Ability to clearly communicate testing results and quality recommendations.
- Demonstration of stakeholder management and documentation skills.
Performance Measurement
Performance will be evaluated based on:
- Volume and quality of testing completed.
- Effectiveness and completeness of test plans and test cases.
- Quality of defect identification, documentation, and resolution tracking.
- Timely delivery against milestones and project objectives.
- Proactive communication regarding risks, blockers, and dependencies.
- Ability to synthesize testing data into actionable recommendations.
- Contributions to process improvements, automation opportunities, and quality enhancements.
Key Responsibilities
Content Testing & Quality Assurance
- Develop and execute test plans for templates, prompt templates, AI-generated content, and other Microsoft 365 content experiences.
- Perform manual and automated testing across Word, Excel, PowerPoint, Create, and Copilot experiences.
- Validate content quality, functionality, usability, accessibility, and adherence to design standards.
- Identify, document, reproduce, and track defects through resolution.
- Verify content performance across a variety of customer scenarios and workflows.
AI Content Evaluation
- Evaluate AI-generated outputs against established quality standards and acceptance criteria.
- Assess content accuracy, relevance, formatting, usability, and overall customer experience.
- Provide actionable recommendations to improve content and model performance.
- Support quality evaluation efforts for emerging generative AI experiences.
Test Development & Execution
- Create and maintain test plans, test cases, scripts, validation frameworks, and testing documentation.
- Develop repeatable testing approaches for new content experiences and product features.
- Collaborate with content creators, product teams, designers, and other stakeholders to improve testing coverage and quality.
- Identify opportunities to streamline testing through automation and scalable processes.
Reporting & Recommendations
- Analyze testing results and communicate findings to stakeholders.
- Identify recurring issues, quality trends, and areas for improvement.
- Track testing outcomes and ensure updates are reflected in documentation and production workflows.
- Translate testing insights into clear, actionable recommendations.
Unique Selling Points
- Work on Leading AI and Productivity Experiences
- Contribute to the quality and effectiveness of content experiences used in Word, Excel, PowerPoint, Create, and Copilot.
- Gain Hands-On Exposure to Generative AI
- Develop expertise in evaluating AI-generated content and improving customer experiences within modern productivity platforms.
- Influence Product Quality at Scale
- Own testing strategies, provide impactful recommendations, and help improve content experiences that reach large and diverse user audiences.
- Collaborate Across Multiple Disciplines
- Partner with content, design, product, and engineering stakeholders to improve quality, usability, and customer outcomes.
- Drive Meaningful Business Impact
- Turn testing insights into process improvements and scalable solutions that directly enhance content quality and user satisfaction.
- Provide your feedback on BizChat