Sr. System Development Engineer, AL/ML/Storage server team

Amazon

Denver, CO

JOB DETAILS
SKILLS
ARM (Advanced RISC Machine), Amazon Web Services (AWS), Architectural Services, Artificial Intelligence (AI), Automation, Automation Systems, C Programming Language, C++ Programming Language, Cloud Computing, Code Reviews, Coding Standards, Computer Engineering, Computer Firmware, Computer Maintenance, Computer Software, Continuous Deployment/Delivery, Continuous Integration, Cross-Functional, Debugging Skills, Design Patterns Programming Methodologies, Device Drivers, Diagnostics Solutions/Software, Distributed Computing, Electrical Engineering, Failure Analysis, GPU (Graphics Processing Unit), Go Programming Language (Golang), Hardware Architecture, Hardware Design, Hardware Development, High Reliability, Home Automation, IPMI (Intelligent Platform Management Interface), Identify Issues, Java, Kernel Programming, Linux Drivers, Linux Kernel Drivers, Linux Operating System, Machine Tool, Manufacturing, Metrics, Microprocessor Architecture, Microsoft C# (C Sharp), National Intelligence Council (NIC), Network Operations Center, Onboarding, Operating Systems, Original Design Manufacturer (ODM), PCI Express (PCI-E), Problem Solving Skills, Process Improvement, Product Development, Product Lifecycle, Product/Service Launch, Production Control, Productivity Management, Programming Languages, Python Programming/Scripting Language, Reporting Dashboards, Root Cause Analysis, Ruby, Scalable System Development, Server Architecture, Server Hardware, Server Programming/Applications, Software Design, Software Development, Source Code/Configuration Management (SCM), Systems Engineering, Team Player, Technical Leadership, Telemetry, Test Plan/Schedule, Testability, Testing, Topology, Validation Testing, Vehicle Fleets, Windows PowerShell, x86 Processors
LOCATION
Denver, CO
POSTED
Today

DescriptionWe are seeking an experienced Senior Systems Development Engineer to lead the development of automation software, diagnostic tooling, and fleet health infrastructure for our server platforms. You will work across multiple teams and organizations to build scalable, reliable systems that keep our storage and accelerated (AI/ML) compute fleet healthy — with a vision toward zero-touch operations where automation detects, diagnoses, and resolves issues without human intervention.You will be a technical leader solving complex architectural problems that may not be well-defined in advance. You will own your team's systems, proactively identify deficiencies, write scalable and robust code to solve issues before they impact customers. You will decompose large, difficult server testability, reliability, and diagnosis problems into straightforward tasks and components — leading delivery yourself and through others in parallel — using a combination of hardware, software, system design, processor architecture, diagnostics, and operations knowledge.You will collaborate with a variety of roles (SDEs, SDETs, Mechanical/Electrical/Hardware Engineers, TPMs, Managers, Principals) and organizations through server conception, test validation, qualification, launch, and operations — driving high quality and reliability into current and future designs for AWS server solutions. You will also work closely with ODMs and Design Partners to ensure our tooling, diagnostics, and automation requirements are met throughout the hardware development lifecycle (NPI).Key job responsibilitiesFleet Health & Predictive InfrastructureBuild and own the automation infrastructure responsible for the health of the server fleet across storage and accelerator (AI/ML) compute platformsDesign and implement predictive failure detection systems using telemetry, sensor data, error trending, and log correlation to identify hardware issues before they cause customer impactDrive toward zero-touch operations — building automation that detects, diagnoses, triages, and remediates hardware and software faults without human interventionDevelop monitoring tools, dashboards, and alerting systems to provide real-time visibility into fleet health across lab and production environmentsDefine and track fleet health metrics (failure rates, mean time to detect, mean time to repair, first-time fix rate, predictive accuracy)Debugging & TroubleshootingDebug and resolve complex system-level issues across storage, compute, GPU, networking in production environmentsTroubleshoot Linux boot and runtime failures across x86 and ARM architectures, including PCIe, power, NIC, NVMe, and GPU subsystemsPerform root cause analysis on hardware failures — correlating across firmware, kernel, driver, and physical layer to isolate faultsBuild diagnostic tooling that automates root cause identification and reduces reliance on manual triageImprove manufacturing throughput and yield through test optimizationSystems Development & AutomationLead the definition and development of software, automation, and enabling tools for server hardware programs; track and report progressDesign and build scalable system-level software with focus on durability, availability, security, and diagnosticsDevelop and maintain device drivers for Linux on ARM and x86 architecturesBuild automation solutions using modern programming languages (Python, Ruby, Java, C/C++, etc.)Work with OS internals, storage subsystems, and accelerator/GPU software stacks in Linux-based environmentsBuild, manage, and deploy CI/CD pipelines for rapid deployment of code changes to org-owned and customer-owned systemsCross-Team CollaborationWork across internal HWEng teams to ensure new server hardware addresses data path and control path functionality needed by dependent service teamsWork closely with internal customers to identify early any potential problems onboarding new servers — storage or accelerated compute — into their ecosystemEngage with ODMs and design partners on testability, diagnostic, and automation requirements during hardware design and developmentContribute to server design to improve robustness, testability, diagnosability, and reliabilityPartner with datacenter operations teams to close the loop between field failures and design improvementsA day in the lifeSystems Development Engineers in AWS Hardware Engineering wear many hats. From orchestration tooling development to hardware integration to kernel driver debugging, we dive deep into problems across the breadth of AWS. Our teams are directly responsible for launching and maintaining server hardware in the fleet — including storage servers powering distributed storage platforms and AI/ML accelerator servers with GPUs. Located in Seattle and Cupertino, we work with internal development teams, ODMs, and design partners to deliver servers deployed in datacenters worldwide.Basic Qualifications6+ years of non‑internship professional software development experience6+ years of systems design, software development, operations, automation, and process improvement experience6+ years of designing or architecting (design patterns, reliability and scaling) of new and existing systems experience5+ years of programming with at least one modern language such as C++, C#, Java, Python, Golang, PowerShell, Ruby experienceExperience with Linux/UnixExperience leading the design, build and deployment of complex and performant (reliable and scalable) software solutions in productionPreferred QualificationsKnowledge of engineering practices and patterns for the full software/hardware/networks development life cycle, including coding standards, code reviews, source control management, build processes, testing, certification, and livesite operationsExperience taking a leading role in building complex software or computing infrastructure that has been successfully delivered to customersExperience building predictive failure detection or proactive remediation systems at fleet scaleExperience with Linux kernel driver developmentExperience with storage, compute, GPU/accelerator platforms (NVIDIA), including driver integration, diagnostics, or performance validationExperience with distributed storage systems (block, object, or file)Familiarity with server hardware architecture, BMC/IPMI, firmware, PCIe topology, NVLink, and hardware diagnosticsExperience working with ODMs or hardware design partners through the product development lifecycleExperience building zero‑touch or self‑healing automation for large‑scale infrastructureExperience working in large‑scale datacenter or cloud environmentsTrack record of rapidly coming up to speed on new engineering disciplines and making impactful decisionsExperience with hardware bring‑up, validation, and fleet‑wide deploymentFamiliarity with telemetry pipelines, anomaly detection, and operational metrics at scaleFamiliarity with manufacturing workflows and yield improvement optimizationAmazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.USA, CA, Cupertino - 173,900.00 - 235,200.00 USD annuallyUSA, CO, Denver - 151,200.00 - 204,600.00 USD annuallyUSA, WA, Seattle - 151,200.00 - 204,600.00 USD annually#J-18808-Ljbffr

About the Company

A

Amazon

At Amazon, we don’t wait for the next big idea to present itself. We envision the shape of impossible things and then we boldly make them reality. So far, this mindset has helped us achieve some incredible things. Let’s build new systems, challenge the status quo, and design the world we want to live in. We believe the work you do here will be the best work of your life.

Wherever you are in your career exploration, Amazon likely has an opportunity for you. Our research scientists and engineers shape the future of natural language understanding with Alexa. Fulfillment center associates around the globe send customer orders from our warehouses to doorsteps. Product managers set feature requirements, strategy, and marketing messages for brand new customer experiences. And as we grow, we’ll add jobs that haven’t been invented yet.

It’s Always Day 1
At Amazon, it’s always “Day 1.” Now, what does this mean and why does it matter? It means that our approach remains the same as it was on Amazon’s very first day – to make smart, fast decisions, stay nimble, invent, and stay focused on delighting our customers. In our 2016 shareholder letter, Amazon CEO Jeff Bezos shared his thoughts on how to keep up a Day 1 company mindset. “Staying in Day 1 requires you to experiment patiently, accept failures, plant seeds, protect saplings, and double down when you see customer delight,” he wrote. “A customer-obsessed culture best creates the conditions where all of that can happen.” You can read the full letter here

Our Leadership Principles
Our Leadership Principles help us keep a Day 1 mentality. They aren’t just a pretty inspirational wall hanging. Amazonians use them, every day, whether they’re discussing ideas for new projects, deciding on the best solution for a customer’s problem, or interviewing candidates. To read through our Leadership Principles from Customer Obsession to Bias for Action, visit https://www.amazon.jobs/principles
COMPANY SIZE
10,000 employees or more
INDUSTRY
Retail
FOUNDED
1994
WEBSITE
http://Amazon.com/militaryroles