Mechanical Commissioning Engineer / Project Manager (Data Centers) Jobot
Sterling, VA2 days agoRemote
- $165,000–$180,000 Per Year
PeopleNTech LLC
- Alexandria, VA
- 30+ days ago
- $75 Per Hour
Want to know if you’re a fit?
Upload your resume and let our AI show you.
Upload your resume and let our AI show you.
Skills
- AWS Lambdaunmatched
- Access Controlunmatched
- Amazon Simple Storage Service (S3)unmatched
- Amazon Web Services (AWS)unmatched
- Apache Sparkunmatched
- Application Programming Interface (API)unmatched
- Architectural Servicesunmatched
- Best Practicesunmatched
- Business Intelligenceunmatched
- Business Supportunmatched
- Cachingunmatched
- Cisco Unityunmatched
- Cloud Computingunmatched
- Coachingunmatched
- Configuration Managementunmatched
- Continuous Deployment/Deliveryunmatched
- Continuous Integrationunmatched
- Cost Controlunmatched
- Cross-Functionalunmatched
- Data Analysisunmatched
- Data Managementunmatched
- Data Partitioningunmatched
- Data Processingunmatched
- Data Setsunmatched
- Data Storageunmatched
- Data Warehousingunmatched
- Database Extract Transform and Load (ETL)unmatched
- DevOpsunmatched
- Gitunmatched
- GitHubunmatched
- High Reliabilityunmatched
- Identify Issuesunmatched
- Machine Toolunmatched
- Performance Tuning/Optimizationunmatched
- Policy Developmentunmatched
- Product Engineeringunmatched
- Regulatory Complianceunmatched
- Relational Databases (RDBMS)unmatched
- Requirements Managementunmatched
- SQL (Structured Query Language)unmatched
- Scalable System Developmentunmatched
- Source Code/Configuration Management (SCM)unmatched
- Streaming Technologyunmatched
- Technical Leadershipunmatched
Description
Job Title: Databricks Data Engineer with DevOps Skills
Location : Los Angeles CA (Hybrid)
Hire type : FTE / CTH
Rate : $75/hr
Salary : $130K
Indent : SF_OP_185860-2-1
Job Summary
We are looking for an experienced Databricks Data Engineer with strong DevOps expertise to join our data engineering team. The ideal candidate will design, build, and optimize large-scale pipelines on the Databricks Lakehouse Platform on AWS, while driving automated CI/CD and deployment practices. This role requires strong skills in PySpark, SQL, AWS cloud services, and modern DevOps tooling. You will collaborate closely with cross-functional teams to deliver scalable, secure, and high-performance data solutions.
Must Demonstrate (Critical Skills & Architectural Competencies)
Key Responsibilities
1. Data Pipeline Development
2. DevOps & CI/CD
3. Collaboration & Business Support
4. Performance & Optimization
5. Governance & Security
Technical Skills
Preferred Experience
Certifications (Optional)
Location : Los Angeles CA (Hybrid)
Hire type : FTE / CTH
Rate : $75/hr
Salary : $130K
Indent : SF_OP_185860-2-1
Job Summary
We are looking for an experienced Databricks Data Engineer with strong DevOps expertise to join our data engineering team. The ideal candidate will design, build, and optimize large-scale pipelines on the Databricks Lakehouse Platform on AWS, while driving automated CI/CD and deployment practices. This role requires strong skills in PySpark, SQL, AWS cloud services, and modern DevOps tooling. You will collaborate closely with cross-functional teams to deliver scalable, secure, and high-performance data solutions.
- Designing and implementing Databricks-based Lakehouse architectures on AWS
- Clear separation of compute vs. serving layers
- Ability to design low-latency data/API access strategies (beyond Spark-only patterns)
- Strong understanding of caching strategies for performance and cost optimization
- Data partitioning, storage optimization, and file layout strategy
- Ability to handle multi-terabyte structured or time-series datasets
- Skill in requirement probing, identifying what matters architecturally
- A player-coach mindset: hands-on engineering + technical leadership
Key Responsibilities
1. Data Pipeline Development
- Design, build, and maintain scalable ETL/ELT pipelines using Databricks on AWS.
- Develop high-performance data processing workflows using PySpark/Spark and SQL.
- Integrate data from Amazon S3, relational databases, and semi/non structured sources.
- Implement Delta Lake best practices including schema evolution, ACID, OPTIMIZE, ZORDER, partitioning, and file-size tuning.
- Ensure architectures support high-volume, multi-terabyte workloads.
- Implement CI/CD pipelines for Databricks using Git, GitLab, GitHub Actions, or AWS-native tools.
- Build and manage automated deployments using Databricks Asset Bundles.
- Manage version control for notebooks, workflows, libraries, and environment configuration.
- Automate cluster policies, job creation, environment provisioning, and configuration management.
- Support infrastructure-as-code via Terraform (preferred) or CloudFormation.
- Work with data analysts and BI teams to prepare curated datasets for reporting and analytics.
- Collaborate closely with product owners, engineering teams, and business partners to translate requirements into scalable implementations.
- Document data flows, technical architecture, and DevOps/deployment workflows.
- Tune Spark clusters, workflows, and queries for cost efficiency and compute performance.
- Monitor pipelines, troubleshoot failures, and maintain high reliability.
- Implement logging, monitoring, and observability across workflows and jobs.
- Apply caching strategies and workload optimization techniques to support low-latency consumption patterns.
- Implement and maintain data governance using Unity Catalog.
- Enforce access controls, security policies, and data compliance requirements.
- Ensure lineage, quality checks, and auditability across data flows.
- Strong hands-on experience with Databricks, including:
- Delta Lake
- Unity Catalog
- Lakehouse Architecture
- Delta Live Pipelines
- Databricks Runtime
- Table Triggers
- Databricks Workflows
- Proficiency in PySpark, Spark, and advanced SQL.
- Expertise with AWS cloud services, including:
- S3
- IAM
- Glue / Glue Catalog
- Lambda
- Kinesis (optional but beneficial)
- Secrets Manager
- Strong understanding of DevOps tools:
- Git / GitLab
- CI/CD pipelines
- Databricks Asset Bundles
- Familiarity with Terraform is a plus.
- Experience with relational databases and data warehouse concepts.
- Knowledge of streaming technologies like Structured Streaming/Spark Streaming.
- Experience building real-time or near real-time pipelines.
- Exposure to advanced Databricks runtime configurations and performance tuning.
- Databricks Certified Data Engineer Associate / Professional
- AWS Data Engineer or AWS Solutions Architect certification
Numbers & Facts
| Location | Alexandria, VA |
Similar Jobs
- New!
- New!
Lead Specialist, Federal Data Engineer KPMG
Washington, DC2 days ago 
Lead Data Scientist Calibre Systems
Alexandria, VA14 days ago- New!
Commissioning Project Engineer (Data Centers) Jobot
Sterling, VA2 days agoRemote- $90,000–$160,000 Per Year
Data Storage Engineer Leidos
Poolesville, MD30+ days ago- $107,900–$195,050
- New!
Project Executive – BAS/BMS - Hyperscale Data Center projects Jobot
Manassas, VA2 days ago- $200,000–$225,000 Per Year