Job Description
Position Title: Databricks Data Engineer
Location: Remote (Harrisburg, PA)
Duration: Long Term Contract
Role and responsibilities:
Client is seeking a Databricks Engineer to lead the design and implementation of a scalable Sales Data Platform as part of the OneData initiative on Databricks running on AWS. The role is hands-on and architecture-driven, focused on data ingestion, transformation, modeling, and optimization using modern lakehouse patterns.
Basic Qualifications:
- 8 years in Data Engineering / Data Architecture roles
- 4 years hands-on experience with Databricks
- Strong expertise in PySpark & Spark SQL
Required Skills:
- Expertise in dbt and Delta Lake
- Strong knowledge of AWS cloud services (S3, IAM, Glue, CloudWatch)
- Experience in Sales data domain (orders, revenue, pricing, customers, products)
- Understanding of Fact & dimension modeling and Slowly Changing Dimensions (SCD Type 1 / 2)
- Experience handling high-volume, high-velocity datasets
Preferred Qualifications:
- Experience with enterprise OneData / Data Mesh programs
- Exposure to real-time or near-real-time ingestion patterns
- Experience integrating CRM / Sales systems (e.g., Salesforce, SAP Sales data)
- AWS certifications or Databricks certifications
Job description:
- Define end-to-end lakehouse architecture on Databricks (AWS) for Sales data domains
- Design medallion architecture (Bronze / Silver / Gold) aligned with OneData standards
- Establish data modeling standards for Sales facts, dimensions, hierarchies, and aggregations
- Define scalable ingestion patterns for batch and incremental loads
- Drive performance, scalability, and cost optimization best practices
- Build and guide development of PySpark-based data pipelines in Databricks
- Implement Delta Lake features including ACID transactions, schema evolution, and versioning
- Design and optimize large-scale joins, aggregations, and window functions
- Implement CDC and incremental processing using watermarking and change detection
- Ensure idempotent, restartable, and fault-tolerant pipelines
- Architect solutions using AWS services such as Amazon S3, IAM, AWS Glue, and CloudWatch
- Optimize Databricks cluster configurations
- Implement secrets management and secure connectivity patterns
- Define and implement data quality checks and validations
- Ensure data lineage and metadata capture
- Implement error handling, auditing, and reconciliation frameworks
- Support data governance and access control requirements
- Implement CI/CD pipelines for Databricks notebooks and jobs
- Enforce code versioning, reviews, and deployment standards
- Design monitoring, alerting, and SLA tracking for pipelines
- Support production stabilization and performance tuning
- Act as technical lead / mentor for Databricks data engineers
- Collaborate with source system teams, data consumers, and cloud/platform teams
- Translate business requirements into robust technical designs
Numbers & Facts
| Location | Chicago, IL (Remote) |
| Industry | Computer/IT Services |
| Salary | $80–$90 Per Hour |
| Company Size | 500 to 999 employees |
| Website | https://www.apolisrises.com/ |
Benefits
Paid Sick Days, Employee Referral Program, Employee Events, Retirement / Pension Plans
About Company
Since 1996, RJT has provided successful SAP, Oracle, and IT consulting solutions and staffing services to clients around the world. The new Apolis brings you the same personalized service fortified with a greater array of IT solutions, global expertise, and cost-management strategies.
We are a global IT consultancy that seamlessly integrates experts and leading-edge solutions into your organization so you can focus on what really matters.