Sr. Product Mgr. Tech, Catalog Experimentation and Impact Measurement

Do you want to shape Amazon"s experimentation and impact-measurement capabilities in the era of Generative AI? As we navigate the rapidly changing landscape of e-commerce, your role will be crucial in shaping how Amazon teams, sellers, and vendors optimize catalog listings for a superior customer experience. Join the Catalog Experimentation and Impact Measurement (CEM) team on our mission to create the best product listings that help customers make well-informed, fast buying decisions. We build the feedback loops that let Large Language Models and Amazon teams enrich product data through experimentation at Amazon scale.

## What you"ll own

In this role, you will lead the conceptualization, delivery, and worldwide adoption of the Catalog Experimentation Platform (CEP). Through experiments, we bring the voice of the customer to our business stakeholders. You will collaborate with economists to help users design experiments correctly, and guide them in interpreting outcomes generated through rigorous statistical analysis. You will advocate for decisions in the best interest of our customers even when the outcomes are unpleasant for the experimenters.

A central part of your charter is impact measurement. Measuring the impact of every initiative that intends to improve product data quality is crucial for prioritization decisions and for fine-tuning each initiative"s strategy. CEM operates a two-part measurement approach: one that estimates how improvements in product data quality translate into incremental sales and customer benefit, and another through which partner teams run controlled experiments while the team reviews their design and aggregates results across the organization. You will collaborate with economists to identify and apply the best possible causal-inference methodology for each initiative, and you will extend the scope of the platform to make it the go-to destination at Amazon for measuring the impact of catalog improvement programs.

Finally, to make experimentation and impact measurement work at Amazon scale without compromising reliability, you will collaborate with your engineering counterparts to build systems for automation, self-service mechanisms, and guardrails.

## Who you"ll work with

You will partner closely with economists and scientists (for experiment design and causal inference), engineering counterparts (for automation, self-service, and guardrails), and the partner and enrichment teams across Amazon whose catalog programs depend on trustworthy impact measurement.

Key job responsibilities

1. Work backwards from the customer experience to define strategy and build a product roadmap for the Catalog Experimentation Platform (CEP), used to run A/B experiments on catalog listings.

2. Launch new products and features by partnering with CEP engineering and science teams.

3. Drive adoption by partnering with the teams building large language models for the catalog and with business stakeholders across different product categories.

4. Partner with economists and other product managers on the team to develop frameworks and shape how business stakeholders and leadership think about impact-measurement methodologies.

5. Represent CEP in Amazon-wide forums on experimentation and impact measurement.

A day in the life

Learn about the strengths and challenges of experimentation platforms used across the industry. Develop product strategy - positioning, go-to-market, and competitive advantage - in collaboration with your engineering and science counterparts. Track key product metrics and communicate wins, misses, and learnings to your key stakeholders. Conduct office hours to guide experimentation users. Communicate strategy and key updates to org leadership through monthly flashes and business-review forums.

About the team

Amazon"s customers rely on the completeness, consistency, and correctness of Amazon"s product data to make well-informed purchase decisions, and improving the quality of that product data is a continuous process. Amazon Selection Catalog Systems (ASCS) is building the Catalog Experimentation and Impact Measurement platform (CEM), which enables teams across Amazon to make data-driven decisions about which product-data changes will simplify and improve the customer experience. It empowers its users to remove personal bias and let data guide decision-making by testing their hypotheses against a representative sample of a broader population. Not least, CEM helps large language models draw causal inferences between catalog-quality improvement actions and customer-experience metrics.