Databricks Data Engineer

Join a Global Leader in Workforce Solutions – Net2Source Inc.
Who We Are
Net2Source Inc. isn't just another staffing company, we're a powerhouse of innovation, connecting top talent with the right opportunities. Recognized for 300% growth in the past three years, we operate in 34 countries with a global team of 5,500+. Our mission? To bridge the talent gap with precision—Right Talent. Right Time. Right Place. Right Price.

Role Name: Databricks Data Engineer
Work site: Harrisburg, PA (Remote)
Duration: 12+ Months

Job Description:

Databricks Engineer – OneData for Quality

Role Summary

We are seeking a Databricks Engineer to lead the design and implementation of a scalable Sales Data Platform as part of the OneData initiative on Databricks running on AWS.
The role is hands-on and architecture-driven, focused on data ingestion, transformation, modeling, and optimization using modern lakehouse patterns.

The architect will work closely with data engineers, source system teams, and downstream consumers to deliver high-quality, governed, and performance-optimized sales datasets.

Key Responsibilities

Architecture & Design
• Define end-to-end lakehouse architecture on Databricks (AWS) for Sales data domains
• Design medallion architecture (Bronze / Silver / Gold) aligned with OneData standards
• Establish data modeling standards for Sales facts, dimensions, hierarchies, and aggregations
• Define scalable ingestion patterns for batch and incremental loads
• Drive performance, scalability, and cost optimization best practices

Data Engineering & Implementation
• Build and guide development of PySpark-based data pipelines in Databricks
• Implement Delta Lake features:
• ACID transactions
• Schema evolution & enforcement
• Time travel & versioning
• Design and optimize large-scale joins, aggregations, and window functions
• Implement CDC and incremental processing using watermarking and change detection
• Ensure idempotent, restartable, and fault-tolerant pipelines

AWS & Platform Integration
• Architect solutions using AWS services:
• Amazon S3 (data lake storage)
• IAM (security & access control)
• AWS Glue / Glue Catalog
• CloudWatch (monitoring & logging)
• Optimize Databricks cluster configurations (job vs all-purpose clusters)
• Implement secrets management and secure connectivity patterns

Data Quality, Governance & Reliability
• Define and implement data quality checks and validations
• Ensure data lineage and metadata capture
• Implement error handling, auditing, and reconciliation frameworks
• Support data governance and access control requirements

DevOps & Operational Excellence
• Implement CI/CD pipelines for Databricks notebooks and jobs
• Enforce code versioning, reviews, and deployment standards
• Design monitoring, alerting, and SLA tracking for pipelines
• Support production stabilization and performance tuning

Collaboration & Leadership
• Act as technical lead / mentor for Databricks data engineers
• Collaborate with:
• Source system teams (Sales, CRM, ERP)
• Data consumers (analytics, downstream apps)
• Cloud/platform teams
• Translate business requirements into robust technical designs

Required Skills & Experience

Core Technical Skills
• 8+ years in Data Engineering / Data Architecture roles
• 4+ years hands-on experience with Databricks
• Strong expertise in PySpark & Spark SQL

• Strong expertise in dbt

• Deep experience with Delta Lake
• Strong knowledge of AWS cloud services (S3, IAM, Glue, CloudWatch)

Data Engineering Expertise
• Sales data domain experience (orders, revenue, pricing, customers, products)
• Strong understanding of:
• Fact & dimension modeling
• Slowly Changing Dimensions (SCD Type 1 / 2)
• Large-scale data processing patterns
• Experience handling high-volume, high-velocity datasets

Platform & Operational Skills
• Databricks job orchestration and scheduling
• Cluster sizing and performance tuning
• CI/CD for data platforms
• Strong troubleshooting and debugging skills

Nice-to-Have
• Experience with enterprise OneData / Data Mesh programs
• Exposure to real-time or near-real-time ingestion patterns
• Experience integrating CRM / Sales systems (e.g., Salesforce, SAP Sales data)
• AWS certifications or Databricks certifications

Why Work With Us?
We believe in more than just jobs—we build careers. At Net2Source, we champion leadership at all levels, celebrate diverse perspectives, and empower you to make an impact. Think work-life balance, professional growth, and a collaborative culture where your ideas matter.
Our Commitment to Inclusion & Equity
Net2Source is an equal opportunity employer, dedicated to fostering a workplace where diverse talents and perspectives are valued. We make all employment decisions based on merit, ensuring a culture of respect, fairness, and opportunity for all, regardless of age, gender, ethnicity, disability, or other protected characteristics.
Awards & Recognition
• America's Most Honored Businesses (Top 10%)
• Fastest-Growing Staffing Firm by Staffing Industry Analysts
• INC 5000 List for Eight Consecutive Years
• Top 100 by Dallas Business Journal
• Spirit of Alliance Award by Agile1
Ready to Level Up Your Career?
Click Apply Now and let's make it happen.

Best regards,
Juan C. Matus 
Sr. Customer Success Manager