Sr Data Engineer - Data Platform
D&H is growing! Join 100+ year old Employee-Owned technology distributor, offering end-to-end solutions for today''s resellers, retailers, and the clients they serve across the SMB and Consumer markets.
- We are empowered by our employee Co-Owners who provide the industry's best service, and we promote a collaborative culture.
- We offer an Employee Stock Ownership Plan, 401k, Paid Time Off, Medical, Prescription, Dental and Vision benefits as well as Gym Reimbursement, Work from Home Reimbursement, Employee Purchase Program, Tuition Assistance and much more!
- As a D&H Co-Owner you receive numerous discounts on services.
- We feel strongly about giving back to the community and promoting sustainable, eco-friendly business practices.
Summary:
We are seeking a Sr. Data Engineer - Data Platform for our Data Science & Insights Department.
The Senior Data Engineer - Data Platform will help build D&H''s next-generation enterprise data platform supporting the modernization of enterprise analytics. This role designs, builds, and operates scalable, production-grade data pipelines across Databricks, SAP data environments, and other cloud and enterprise sources. Candidates are not expected to bring deep expertise in every technology within D&H''s target architecture. Successful candidates will combine strong foundational data engineering experience with deep expertise in either modern Databricks/Spark engineering or SAP data engineering, and the ability and interest to develop broader capability across the integrated platform.
Responsibilities:
- Design, develop, test, deploy, and support scalable data pipelines across SAP, Databricks, and related cloud and enterprise environments.
- Develop production-grade ingestion and transformation workloads using SQL and appropriate distributed data-processing technologies.
- Apply reusable engineering patterns for orchestration, data quality, observability, monitoring, error handling, and recovery.
- Collaborate with SAP, Data Platform, Infrastructure, Security, Architecture, Analytics, and business teams to deliver reliable end-to-end data solutions.
- Contribute to lakehouse and medallion architecture patterns and the development of governed, reusable data products.
- Optimize data workloads for performance, scalability, reliability, maintainability, and cost efficiency.
- Follow engineering practices for source control, automated testing, CI/CD, deployment, documentation, and environment promotion.
- Support migration from legacy EDW/ETL platforms to modern cloud-based architectures while validating completeness and data integrity.
- Deliver trusted and certified datasets for enterprise reporting, analytics, planning, and AI use cases.
- Participate in design reviews, troubleshooting, production support, knowledge transfer, and continuous improvement of team standards.
- Develop broader proficiency across D&H's integrated SAP and Databricks platform through applied project work, mentoring, and training.
First-Year Success:
- Establish and apply production-ready data engineering patterns that support D&H's modern data platform.
- Deliver reliable data pipelines connecting SAP data environments, Databricks, and other enterprise sources.
- Build capabilities for ingestion, transformation, orchestration, monitoring, recovery, and data quality.
- Contribute specialized depth in either Databricks/Spark engineering or SAP data engineering while developing working knowledge of the broader platform.
- Help transition implementation knowledge, technical patterns, and operational responsibility into D&H's internal team.
Requirements:
- 8+ years of progressive experience in data engineering, data integration, ETL/ELT, or enterprise data platform development.
- Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a related field; or an equivalent combination of education, training, and relevant professional experience.
- Strong hands-on experience with either modern Databricks/Spark data engineering or SAP enterprise data engineering, with the ability to contribute across integrated data environments.
- Advanced SQL skills and strong knowledge of data modeling and scalable data-transformation patterns.
- Experience building and supporting production-grade data pipelines, including orchestration, monitoring, data quality, error handling, recovery, and performance optimization.
- Experience working with cloud or distributed data architectures.
- Ability to translate business and technical requirements into scalable, maintainable data solutions.
- Strong problem-solving, collaboration, documentation, and communication skills across technical and business teams.
- Ability to work effectively in a modernization environment where technologies, standards, and operating practices continue to evolve.
Preferred Qualifications:
Depth in one or more of the following areas is preferred. Expertise across all listed technologies is not required.
Databricks and Modern Data Engineering
- Databricks, Apache Spark, PySpark, Spark SQL, DataFrames, or comparable distributed data-processing technologies.
- Delta Lake, Unity Catalog, Databricks Workflows/Lakeflow, Delta Live Tables/Declarative pipelines, or Databricks SQL].
- Lakehouse and medallion architecture patterns.
SAP Data Engineering
- SAP Datasphere, Graphical/SQL views, Analytical Models, Replication/Transformation Flows, SAP HANA Calculation views and/or Stored Procedures within HDI container.
- SAP BW/4HANA, BW Bridge/BW Model Transfer, Data product generator, HANA Calculation Views.
- SAP S/4HANA data structures, CDS Views, SAP ODP data extraction or integration patterns, ODATA APIs.
Cloud and Engineering Practices
- Microsoft Azure, including cloud storage and supporting security, networking, or identity concepts.
- Git/source control, CI/CD, automated testing, automated deployment, and DataOps/DevOps practices.
- Metadata, lineage, role-based access, data quality, monitoring, and observability.
- Streaming or event-driven technologies such as Apache Kafka or comparable platforms.
Analytics Ecosystem
- Semantic modeling and governed data-product development.
- SAP Analytics Cloud, Power BI, Tableau, Cognos, or comparable enterprise analytics platforms.
- Databricks and/or SAP technical certifications are a plus.
EOE