Lead Data Engineer - Experimentation Platform - 1633

Must Have:

• Python
  
• SQL
  
• Data Engineering
  
• ETL / ELT
  
• Apache Spark
  
• Databricks
  
• Snowflake
  
• Apache Kafka
  
• Apache Airflow
  
• Streaming Data Pipelines
  
• Data Modeling
  
• Data Warehousing / Lakehouse
  
• A/B Testing / Experimentation Platforms
  
• CI/CD for Data Pipelines
  
• Data Quality & Data Governance
  
• Cloud Data Platforms
  

Responsibilities:

• Design and build scalable data platforms supporting experimentation and A/B testing.
  
• Develop batch and streaming data pipelines for large-scale user and product datasets.
  
• Build reusable datasets and frameworks for experimentation, analytics, and product measurement.
  
• Design dimensional data models and analytics-ready data products.
  
• Implement automated data quality, validation, monitoring, lineage, and governance.
  
• Build production-grade deployment pipelines with CI/CD and observability.
  
• Partner with Product, Engineering, Data Science, and Analytics teams to deliver scalable data solutions.
  
• Optimize data infrastructure supporting experimentation, personalization, and machine learning workloads.
  
• Mentor engineers and establish best practices for large-scale data engineering.
  

Qualifications:

• Bachelor's degree in Computer Science, Engineering, Mathematics, Statistics, or a related technical field.
  
• 7+ years of experience in data engineering or large-scale data platforms.
  
• Strong experience with distributed data processing and cloud-based data architectures.
  
• Hands-on experience with Python, SQL, Spark, Databricks, Snowflake, Kafka, and Airflow.
  
• Strong understanding of data modeling, ETL/ELT, streaming architectures, and lakehouse concepts.
  
• Experience building experimentation, analytics, personalization, or ML data platforms.
  
• Experience implementing CI/CD, automated testing, monitoring, and data governance.
  
• Strong system design and architecture experience.
  
• Experience mentoring engineers and leading technical initiatives.
  

Nice to Have:

• Experimentation platforms or A/B testing infrastructure.
  
• Causal inference or product analytics experience.
  
• ML feature engineering and model lifecycle pipelines.
  
• Infrastructure automation and observability.
  
• Subscription, streaming media, advertising, or consumer product experience.
  
• MS or PhD in a related technical field.