Data Engineer

California, CA

3 days ago

Job Description

We are seeking an experienced Data Engineer to design, build, and optimize scalable data pipelines and cloud-based data platforms. The ideal candidate will have strong expertise in Python, SQL, Spark, ETL processes, and cloud technologies to support data-driven decision-making and advanced analytics.

Key Responsibilities

Design, develop, and maintain scalable data pipelines and ETL/ELT workflows.
Build and optimize data processing solutions using Python, SQL, and Apache Spark (PySpark).
Develop batch and real-time data ingestion pipelines.
Work with structured and unstructured datasets from multiple data sources.
Design and implement cloud-based data solutions using Azure, AWS, or GCP.
Develop and maintain data models, data lakes, and enterprise data warehouses.
Integrate data from APIs, databases, and streaming platforms.
Optimize data pipeline performance and ensure data quality.
Collaborate with Data Scientists, Business Analysts, and Software Engineers.
Implement CI/CD pipelines and DevOps best practices for data engineering.
Troubleshoot production issues and provide ongoing support.
Ensure compliance with data governance, security, and privacy standards.

Required Skills

5+ years of experience as a Data Engineer.
Strong programming skills in Python.
Excellent SQL skills.
Hands-on experience with Apache Spark / PySpark.
Experience with ETL/ELT development.
Strong knowledge of Data Warehousing concepts.
Experience with Hadoop ecosystem (Hive, HDFS).
Experience with Databricks.
Experience with Apache Kafka.
Hands-on experience with Apache Airflow or similar workflow orchestration tools.
Experience working on Linux/Unix environments.
Knowledge of Git and CI/CD pipelines.

Cloud Skills

Experience with one or more of the following:

Microsoft Azure (ADF, ADLS, Synapse)
Amazon Web Services (AWS)
Google Cloud Platform (GCP)

Preferred Skills

Snowflake
Delta Lake
Docker
Kubernetes
Terraform
Scala or Java
Power BI or Tableau
REST APIs
Agile/Scrum methodology
Experience working with large-scale enterprise data platforms

Numbers & Facts

Location

California, CA

Skills

Agile Programming Methodologiesunmatched
Amazon Web Services (AWS)unmatched
Apacheunmatched
Apache Hadoopunmatched
Apache Hiveunmatched
Apache Kafkaunmatched
Apache Sparkunmatched
Application Programming Interface (API)unmatched
Best Practicesunmatched
Business Analysisunmatched
Cloud Computingunmatched
Computer Programmingunmatched
Continuous Deployment/Deliveryunmatched
Continuous Integrationunmatched
Data Managementunmatched
Data Modelingunmatched
Data Processingunmatched
Data Qualityunmatched
Data Scienceunmatched
Data Setsunmatched
Data Warehousingunmatched
Database Extract Transform and Load (ETL)unmatched
DevOpsunmatched
Dockerunmatched
Ecosystemsunmatched
GCP (Good Clinical Practices)unmatched
Gitunmatched
HDFS (Hadoop Distributed File System)unmatched
Identify Issuesunmatched
Information/Data Security (InfoSec)unmatched
Javaunmatched
Linux Operating Systemunmatched
Maintain Complianceunmatched
Microsoft Windows Azureunmatched
Performance Tuning/Optimizationunmatched
Power BIunmatched
Privacy Controlsunmatched
Python Programming/Scripting Languageunmatched
REST (Representational State Transfer)unmatched
SQL (Structured Query Language)unmatched
Scala Programming Languageunmatched
Scalable System Developmentunmatched
Scrum Project Management and Software Developmentunmatched
Software Engineeringunmatched
Tableauunmatched
Technical Supportunmatched
Unix Operating Systemsunmatched

Level up your application

Professional resume templates

Browse dozens of recruiter approved resume templates, layouts and formats. Choose your favorite and make it your own in minutes.

Free resume templates

Resume Builder

Improve your existing resume or start from scratch and create a standout, ATS-friendly resume. Add job-specific content, download and apply.

Resume Builder