Job Title: Senior Databricks Data Engineer - Remote
Job Description:
We are seeking an experienced Senior Databricks Data Engineer with strong expertise in designing, developing, and optimizing large-scale data engineering solutions on the Databricks platform. The ideal candidate will have extensive hands-on experience in building enterprise-grade data pipelines, processing high-volume datasets, and delivering scalable analytics solutions using PySpark, Spark SQL, and AWS cloud technologies.
Key Responsibilities:
Design, develop, and maintain scalable data engineering solutions using Databricks, PySpark, Spark SQL, and Apache Spark. Build and optimize end-to-end ETL/ELT pipelines integrating Databricks with AWS S3, Snowflake, Airflow, and AWS Glue.
Process and transform large-scale structured and semi-structured datasets to support enterprise reporting, analytics, and regulatory requirements.
Develop reusable frameworks and components for: Data ingestion Data cleansing and transformation Data validation and reconciliation Data enrichment Business rule implementation Implement robust data quality checks, monitoring, and governance controls to ensure data accuracy and reliability.
Perform performance tuning and optimization of Spark applications and Databricks workloads to improve scalability and processing efficiency.
Collaborate with business stakeholders, architects, and analytics teams to understand data requirements and deliver scalable solutions.
Troubleshoot and resolve production data pipeline issues while ensuring high availability and reliability.
Support advanced analytics, customer segmentation, campaign analytics, AML/regulatory reporting, and data validation initiatives.
Required Skills:
Technical Skills 7+ years of hands-on experience with Databricks and modern data engineering platforms.
Experience building and managing ETL/ELT pipelines on cloud platforms.
Strong experience with: AWS S3 Snowflake Apache Airflow AWS Glue Knowledge of data modeling, data warehousing, and big data processing techniques.
Experience working with structured and semi-structured data formats such as JSON, Parquet, Avro, and CSV.
Hands-on experience implementing data quality frameworks and monitoring solutions. Strong SQL and data analysis skills. Preferred Skills Experience with regulatory reporting, AML, compliance, or financial data domains. Exposure to CI/CD and DevOps practices for data engineering.
Experience with Delta Lake, Databricks Workflows, and Unity Catalog.
Knowledge of Agile/Scrum delivery methodologies.
Key Experience Areas Candidates should have demonstrated experience in one or more of the following:
Customer Segmentation Analytics Campaign Analytics AML and Regulatory Reporting Enterprise Data Platforms Production Data Pipeline Support Data Validation and Reconciliation Large-Scale Data Migration and Modernization Programs
Numbers & Facts
Location
Aliso Viejo, CA (Remote)
Salary
$54.28–$58.46 Per Hour
Skills
Agile Programming Methodologiesunmatched
Amazon Simple Storage Service (S3)unmatched
Amazon Web Services (AWS)unmatched
Analysis Skillsunmatched
Apacheunmatched
Apache Avrounmatched
Apache Sparkunmatched
Architectural Analysisunmatched
Big Dataunmatched
Business Architectureunmatched
Campaignsunmatched
Cisco Unityunmatched
Cloud Computingunmatched
Continuous Deployment/Deliveryunmatched
Continuous Integrationunmatched
Data Analysisunmatched
Data Formatsunmatched
Data Managementunmatched
Data Migrationunmatched
Data Modelingunmatched
Data Processingunmatched
Data Qualityunmatched
Data Setsunmatched
Data Warehousingunmatched
Database Extract Transform and Load (ETL)unmatched
DevOpsunmatched
Financial Complianceunmatched
High Availabilityunmatched
High Reliabilityunmatched
Identify Issuesunmatched
JSONunmatched
Market Segmentationunmatched
Money Launderingunmatched
Performance Tuning/Optimizationunmatched
Production Supportunmatched
Quality Monitoringunmatched
Reconciliationunmatched
Regulatory Reportsunmatched
Regulatory Requirementsunmatched
Requirements Managementunmatched
SQL (Structured Query Language)unmatched
Scalable System Developmentunmatched
Scrum Project Management and Software Developmentunmatched
Snowflake Schemaunmatched
Structured Dataunmatched
🎯
Be found by employers
5,500+ employers search our resume database daily. Add yours to get found by recruiters looking for candidates like you.
Level up your application
Professional resume templates
Browse dozens of recruiter approved resume templates, layouts and formats. Choose your favorite and make it your own in minutes.