Data Engineer II, Data Engineer,Data Center Capacity Delivery

AWS Data Center Capacity Delivery (DCCD) is looking for a Data Engineer to support data center construction globally. We work on the most challenging problems, with thousands of variables impacting the data center delivery - and we're looking for talented people who want to help.

You'll join a diverse team of software, hardware, and network engineers, construction specialists, security experts, operations managers, and other vital roles. You'll collaborate with people across AWS to help us deliver the highest standards for safety and security while providing seemingly infinite capacity at the lowest possible cost for our customers. You'll experience an inclusive culture that welcomes bold ideas and empowers you to own them to completion.

We're looking for Data Engineer to help us grow our Data Lake and Data Warehouse Systems, which is being built using a serverless architecture, with 100% native AWS components including Redshift Spectrum, Athena, S3, Lambda, Glue, EMR, Kinesis, SNS, CloudWatch and more! We own a world-class data lake that is used to drive multi-billion dollar decisions on a regular cadence and we"re looking to improve on filling the lake quickly, with as little human intervention needed and democratize the data in the lake.

Our Data Engineers build the ETL and analytics solutions for our internal customers to answer questions with data and drive critical improvements for the business. Our Data Engineers use best practices in software engineering, data management, data storage, data compute, and distributed systems. We are passionate about solving business problems with data!

Key job responsibilities

• Design and implement scalable, fault-tolerant data pipelines using AWS technologies and internal Amazon tools to extract, transform, and load data from multiple sources leveraging and implementing AI solutions as required.
• Collaborate cross-functionally with BIEs, Data Scientists, PMs, and SDEs to understand data requirements and deliver customized data solutions.
• Automate infrastructure deployment with CI/CD pipelines and ensure streamlined processes for deployment and maintenance.
• Ensure data quality through robust validation, cleansing, and deduplication techniques.
• Implement data governance standards, including access control, encryption, data retention, deletion policies, and audit mechanisms to ensure compliance and security.
• Continuously improve and optimize data pipelines and infrastructure, staying up to date with emerging technologies and implementing automation and monitoring tools.
• Build a scalable and reliable data platform supporting analytics for intuitive, self-service data products.
• Write high quality code and build scalable applications that interface with critical services and APIs to extract and process unstructured data
• Work with a range of data technologies, including Python, EMR, Spark, Iceberg, Airflow, and many AWS data services like Glue, Athena, Redshift to create end-to-end pipelines that consolidate data from disparate systems.

About the team

DCCD -CAT is a central data and analytics team within the DCCD tooling org that plays a pivotal role in supporting analytics for data center construction space suporting cost,controls and commissioning domains. We own data platform, reporting, dashboards, measurement, and analytical solutions for DCCD org.