Data Engineer – Autonomous Vehicle

Data Engineer with strong AWS + Python/SQL, who builds scalable pipelines for ML training and has experience with large sensor or simulation datasets.

Experience Expectations
• Expectation on the type of experience.
• They want to know if someone with consulting experience who has done small projects
at different enterprises is good enough.
• Automotive-specific knowledge is not necessarily required.
Role Context: Training World Models
• The goal is to train world models for autonomous driving applications.
• This involves forward prediction models for sensor data.
• They need to ingest large amounts of observational real-world data.
• They need to bring data into a compute environment to train models.
• They need to align the data schema across disparate open source datasets.
• The goal is to "bring the data into our compute infrastructure in a way that's usable for
research."
Data Processing and AWS
• The individual will need to do data processing to normalize data into a medium that is
compatible with ML training tooling.
• This will be done primarily in AWS.
• There are any specific AWS tools that are must-haves.
• They also ask if they are looking for someone to work with tooling that's agnostic and
not necessarily specific to AWS.

Tools and Technologies
• The tools used are largely AWS tools.
• Experience with Azure is not necessary.
Data Quality and Schema Enforcement
• The research org is not as regimented as some of the more
production-oriented AP companies.
• The ingestion and cleanup of data is the primary ask.

Collaboration and Communication
• The idea is to find someone to assist the researchers, freeing them up to focus on model
development and architecture.
• The researchers would work with the contractor to see what's feasible and useful.
• Strong communicator who thrives in ambiguous, research-driven environments, takes ownership of data systems, and collaborates effectively with ML and research teams.

Nice-to-have experience includes autonomous vehicle or robotics sensor data, simulation pipelines, ML training data support, dataset versioning, and working closely with research teams.