Data Engineer, Amazon Security Experience

The Amazon Security Experiences (ASE) organization is the data intelligence backbone of Amazon Security. We build and operate the data platform that powers security insights, builder experience analytics, and tooling decisions across Amazon"s security ecosystem. Every insight surfaced to security leadership, from builder productivity trends to security issue quality metrics, flows through systems our team designs, builds, and maintains.

We operate at scale, processing large volumes of security and operational data daily through hundreds of pipelines consisting of transformations, reporting queries, ingestions, and infrastructure management scripts. Our engineers work directly with source systems across AWS and SDO (Security, Developer, and Operations) data domains to procure data, convert it into structured formats, build large-scale processing pipelines, design analytical data models, and maintain infrastructure with the highest security and compliance standards.

We are seeking a Data Engineer I to join our Security Data Intelligence (SDI) team. In this role, you will build and maintain data pipelines that power security analytics across Amazon. Working with well-defined requirements and guidance from senior engineers, you will create physical data models, optimize data pipelines for defined datasets, and deliver reliable, well-documented solutions on schedule. You will partner with teams across ASE, BizOps, MapSec, and other security organizations to implement data ingestion flows and support analytical models that serve the broader Amazon Security organization.

Key job responsibilities

Learn and apply team-established data processing guidelines and best practices when building and testing pipelines. Measure and improve dataset quality across owned pipelines, identifying issues and proposing solutions to the team. Flag limitations or opportunities in data processing tools to senior engineers for evaluation.

Learn and understand the team"s existing data architecture, including how security data flows through ingestion, transformation, and serving layers. Build data solutions at the component level within the established team data architecture, following defined logical data models. Create physical data models for assigned datasets, applying appropriate persistence and integration techniques. Understand the implications of implementation decisions (e.g., normalization, distribution) and seek guidance from senior engineers on trade-offs.

Write secure, stable, testable, and maintainable code with minimal defects, following team coding standards. Clearly document solutions for ease of use and maintainability by teammates and data consumers. Get all data models, pipeline designs, and code reviewed by senior engineers before deployment. Test code thoroughly and understand the maintenance characteristics, runtime properties, and dependencies of delivered solutions.

Build and maintain data pipeline infrastructure for assigned datasets using team-established patterns and tools. Apply basic database optimization techniques such as partitioning, distribution, and indexing as directed. Understand the underlying system infrastructure that data solutions depend on, including differences between data persistence solutions. Follow team guidelines for infrastructure management, security compliance, and automation practices.

Solve straightforward, well-defined problems with guidance from senior engineers, for example, building ingestion pipelines for a new security data source where the schema and requirements are defined, or troubleshooting data quality issues in existing datasets by researching root causes and thoroughly resolving defects. Handle problems even outside your own domain by either resolving them or ensuring a clear hand-off to the right owner.

Take well-defined requirements and deliver solutions on schedule within assigned scope. Participate in team design, scoping, and prioritization discussions to understand project context. Manage your own task execution effectively, seeking guidance when blocked or uncertain. Communicate progress and dependencies clearly to your team lead and senior engineers.

Build good working relationships with teammates, peer engineers, and data consumers across ASE and SDO. Partner effectively with Business Analysts, Data Scientists, and peer SDEs to understand data requirements. Help train and mentor DE interns when applicable. Participate in the interview process. Recognize discordant views and participate in constructive dialogue with peers.

Follow team-established best practices for data discovery, naming conventions, and data security. Classify, store, and handle data per Amazon security and compliance policies. Ensure proper access control, data integrity, and auditability for all produced datasets. Track security risks in data handling, mitigate issues, and escalate in a timely manner.

Identify data quality issues and pipeline deficiencies in owned components and propose solutions to the team. Participate in code reviews, design discussions, and team planning sessions. Make small improvements to team development processes when directed. Actively seek knowledge of team data architecture and apply learnings to improve your data solutions.

Participate in on-call rotation and respond to pipeline incidents following established runbooks and escalation procedures. Understand the maintenance characteristics, runtime properties, and dependencies of team data systems. Contribute to monitoring and alarming setup for owned pipelines with guidance from senior engineers. Document incident resolutions and contribute to runbook updates for owned components.