Applied Science Manager, Artificial General Intelligence , Quality Automation

The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Science Manager with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to lead a team ensuring the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems.

Key job responsibilities:

• As part of the AGI team, an Applied Science Manager will lead and mentor a team of Applied Scientists who develop comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows. • The manager will guide the team in designing auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that align with core scientist team developing Amazon Nova models. • The Applied Science Manager will oversee expert-level manual audits, meta-audits to evaluate auditor performance, and provide coaching to uplift overall quality capabilities across the team. • The manager will lead research in areas related to HIL data impact to LLM models, and define utility measurement strategies for data generated by AGI-DS for Nova models.

The Applied Science Manager will be responsible for:

• Recruiting, hiring, and developing team members • Conducting performance reviews • Setting clear expectations and growth plans • Fostering a culture of scientific excellence and innovation • Communicating with senior leadership, cross-functional technical teams, and customers to collect requirements, describe product features and technical designs, and articulate product strategy

A day in the life:

An Applied Science Manager with the AGI team will:

• Lead quality solution design • Guide root cause analysis on data quality issues • Drive research into new auditing methodologies • Find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards

The manager will work closely with:

• Talented engineers • Domain experts • Vendor teams • To put quality strategies and automated judging systems into practice

The manager will also:

• Conduct regular 1:1s with team members • Provide mentorship and coaching • Ensure the team delivers high-impact results aligned with organizational goals