Sr Product Manager, Technical, Core Shopping Data Science

Amazon cannot fix a customer experience problem it cannot measure. Our team"s job is to make shopping quality measurable: to find the defects customers actually encounter, quantify how often they encounter them, and put a number in front of leadership that teams are willing to be held to. Today that capability is strongest on the search results page, where we run a portfolio of quality metrics covering areas such as relevance, duplication, and brand quality, backed by a mix of human annotation and LLM based measurement.

We are hiring a Sr. Product Manager - Technical to own the quality bar for core shopping: the standard that defines what counts as a customer visible defect, how precisely it must be measured, and what evidence is required before a number can be trusted. You will drive alignment on that standard across the organizations whose experiences it judges. Owning the bar is the job. Measurement and inspection are how you enforce it: you will own the quality metric portfolio for the search page, the inspection mechanisms that surface the defects we are not yet measuring, and the scalable mechanisms (human annotation, LLM based evaluation, and the audit loops that keep both honest) that hold the system together as coverage grows. You will expand that coverage beyond search to other core shopping surfaces, including the Homepage and Detail Page.

Key job responsibilities

Define the quality bar:

• Own the standard that every metric definition, annotation SOP, and automated measurement approach must satisfy before it can be trusted or published. Scientists, engineers, and annotation partners build to that bar.
• Own the portfolio of shopping quality metrics against that bar, covering areas such as relevance, duplication, and brand quality, and grow that portfolio as new defect classes are identified.

Own the inspection mechanisms

• Own how we find quality defects, not just how we count the ones we already know about: sampling strategy, audit cadence, anecdote review, and systematic inspection of pages customers actually saw.
• Turn qualitative signal into quantified metrics. Work with UX Research and leadership to convert customer anecdotes and research findings into defect definitions that can be sampled, scored, and tracked.
• Own the audit loop and the SOP bar. Specify what gets audited and against which criteria, review findings, and require the resulting corrections in annotation SOPs or LLM prompts. When a measurement approach systematically misses a defect class, you own defining what the corrected standard is, and the team builds to it.

Drive quality measurement into online systems

• Drive the agenda to move quality measurement from offline, after the fact reporting into online systems, so that quality signals are available where experiences are ranked and served rather than only in a periodic report.
• Partner with central platform teams to embed quality metrics into online evaluation and serving paths.
• Make the case for where online quality signals change decisions, including experimentation guardrails, faster detection of quality regressions, and closed loop correction of defective experiences.

Own the roadmap

• Own the multi year roadmap for shopping quality measurement and inspection: which defects to measure next, which surfaces to expand to, and what each expansion unblocks for the business.
• Extend the charter beyond search to other core shopping pages, starting with Homepage and Detail Page. For surfaces with no measurement today, define the defect taxonomy from scratch: what a quality defect is on that page, how it is sampled, how it is scored, and how it rolls up.
• Drive alignment across Organic Search, Sponsored Products, Sponsored Brands, and International as definitions evolve, including with the teams whose experiences the metrics judge.

About the team

The mission of the Core Shopping Data Science team is to provide the data-driven foundation for building a world-class shopping experience that maximizes long-term free cash flow by delighting customers. We focus on the long term and big picture to ensure that the full Amazon shopping experience balances strategic trade-offs. We believe data science is the discipline of making smart decisions and we empower feature owners and systems by 1) developing and vending metrics which assess and value customer engagement, 2) building tools and datasets to inject data into the decision-making process, and 3) delivering deep analyses to inform high-touch decisions.