Position Summary
We are seeking Senior Data Engineers with deep hands-on Databricks experience to build and maintain modern data platforms supporting complex enterprise and government programs.
This is a hands-on engineering role. You will design and implement data pipelines and lakehouse solutions, contribute to architectural decisions, review your peers' code, and work directly with clients on technical solutions. You'll partner with architects and the practice lead on standards and direction, and mentor less experienced engineers on the team.
Key Responsibilities
Databricks EngineeringBuild scalable, secure data solutions on the Databricks Lakehouse Platform using Apache Spark, Delta Lake, and cloud-native services.
Implement batch, streaming, ETL, ELT, analytics, and ML-enabled data pipelines.
Apply medallion architecture patterns across ingestion, transformation, orchestration, and consumption layers.
Implement data governance in Unity Catalog — lineage, access controls, auditing, metadata, and secure sharing — following established standards.
Configure clusters, serverless compute, and workloads for performance, reliability, and cost efficiency.
Develop with PySpark, Spark SQL, Python, Delta Live Tables, Structured Streaming, Auto Loader, MLflow, and Databricks Workflows.
Integrate Databricks with Azure, AWS, or Google Cloud services.
Contribute to migrations and modernization from legacy databases, data warehouses, Hadoop, and traditional ETL platforms.
Follow and help improve engineering practices for source control, automated testing, CI/CD, infrastructure as code, monitoring, and production support.
Participate in architecture reviews, code reviews, technical assessments, and root-cause analysis.
Troubleshoot production issues and tune underperforming Spark workloads and queries.
Collaboration and Technical Contribution Mentor junior and mid-level engineers through pairing, code review, and knowledge sharing.
Translate business, functional, and security requirements into technical designs and working code.
Contribute estimates and technical input to project plans, and flag risks and dependencies early.
Document designs, pipelines, and operational runbooks.
Build reusable components, templates, and accelerators the wider team can adopt.
Participate in client meetings as a technical contributor; support proposals, solution estimates, and demonstrations as needed.
Communicate status, blockers, and technical trade-offs clearly to project leads and stakeholders.
| Location | (Remote) |
| Job Type | Contractor |
Required Qualifications Bachelor's degree in computer science, information technology, data engineering, engineering, or a related discipline — or equivalent practical experience.
At least 6 years in data engineering, data architecture, or analytics engineering.
At least 3 years of hands-on experience designing and implementing solutions on Databricks.
Strong working experience with: Databricks Lakehouse Platform
Apache Spark and PySpark
Spark SQL and advanced SQL development
Delta Lake and medallion architecture
Unity Catalog and data governance concepts
ETL and ELT pipeline design
Batch and real-time data processing
Data modeling and data warehousing
Python-based data engineering
Databricks Workflows, Jobs, and cluster management
Experience deploying Databricks solutions on Azure, AWS, or Google Cloud.
Experience with CI/CD, Git-based development, automated testing, and infrastructure as code.
Demonstrated ability to optimize Spark workloads, cluster configurations, query performance, and cloud costs.
Clear written and verbal communication, with the ability to explain technical concepts to non-technical audiences.
Authorized to work in the United States or Canada.
Preferred QualificationsDatabricks Certified Data Engineer Associate or Professional, or Databricks Certified Machine Learning Professional.
Azure, AWS, or Google Cloud associate- or professional-level certification.
Experience in consulting, professional services, systems integration, or managed services.
Experience supporting federal, state, or local government clients.
Familiarity with federal security, privacy, governance, and compliance requirements.
Experience with Azure Data Factory, ADLS, Synapse, AWS Glue, S3, Snowflake, dbt, Kafka, Airflow, or Terraform.
Experience with MLflow, MLOps, generative AI, Databricks Mosaic AI, or vector search.
Experience working on distributed or fully remote engineering teams.
What Success Looks LikePipelines and solutions that are scalable, secure, reliable, and well documented.
Committed work delivered on schedule, with risks raised before they become escalations.
Code and designs that raise the quality bar for the team.
Efficient use of Databricks consumption and cloud infrastructure.
Adoption of shared architectures and reusable components across engagements.
Browse dozens of recruiter approved resume templates, layouts and formats. Choose your favorite and make it your own in minutes.
Free resume templatesImprove your existing resume or start from scratch and create a standout, ATS-friendly resume. Add job-specific content, download and apply.
Free resume builder