Data Pipeline Development, Enhancement & Maintenance
Design, build, and maintain scalable ETL/ELT pipelines.
Automate data ingestion from multiple data sources.
Ensure reliable, high-performance, and scalable data movement.
Develop and optimize PySpark-based data processing frameworks.
Build and manage Airflow workflows and orchestration pipelines.
Work with Hive and Kafka-based batch and streaming data solutions.
Implement data quality, monitoring, governance, and operational controls.
Support production incidents, performance tuning, and continuous improvement initiatives
Roles & Responsibilities
The Data Engineer will be responsible for designing, building, governing, and optimizing data platforms, pipelines, and integrations to ensure secure, reliable, and scalable data availability for analytics, reporting, AI/ML, and business operations.
Preferred Experience
Strong Functional Knowledge in Retail (Search & Personalization) Added advantage.
Strong experience with large-scale GCP Data Engineering solutions.
Hands-on experience with Airflow DAG development and support
Experience building and optimizing Spark/PySpark data pipelines.
Exposure to Hive-based data processing and Kafka-driven ingestion frameworks.
Experience supporting analytics, personalization, recommendation, or search platforms is highly desirable
Numbers & Facts
Location
Sunnyvale, CA (Remote)
Salary
$110,000–$150,000
Skills
Apiary/Beekeepingunmatched
Artificial Intelligence (AI)unmatched
Business Operationsunmatched
Cloud Computingunmatched
Continuous Improvementunmatched
Data Analysisunmatched
Data Managementunmatched
Data Processingunmatched
Data Qualityunmatched
Database Extract Transform and Load (ETL)unmatched
GCP (Good Clinical Practices)unmatched
Performance Tuning/Optimizationunmatched
Production Supportunmatched
Python Programming/Scripting Languageunmatched
Quality Monitoringunmatched
Retailunmatched
Scalable System Developmentunmatched
🎯
Be found by employers
5,500+ employers search our resume database daily. Add yours to get found by recruiters looking for candidates like you.
Level up your application
Professional resume templates
Browse dozens of recruiter approved resume templates, layouts and formats. Choose your favorite and make it your own in minutes.