Databricks Data Engineering Consultant

Job Title: Databricks Data Engineering Consultant

Location: Hybrid – Indianapolis, IN

Job Type: Full-Time

About SDG Group

SDG Group is a global consulting firm specialized in Data, Analytics & AI. We are committed to unlocking organizations’ potential and hidden value by offering in-depth expertise that empowers our clients’ business models to become successful data-driven enterprises.

Innovation is in our DNA. We constantly evolve our value proposition through cutting-edge laboratories, advanced analytics, and AI technologies — enabling our clients to accelerate their data transformation journeys.

SDG Group operates on a model of Triple Expertise — combining deep industry domain knowledge, technological proficiency, and strategic partnerships with leading software providers such as Snowflake, Matillion, Databricks, and Microsoft.

With proven experience across 700+ clients, we deliver Data & AI solutions tailored to diverse industries. Our unique IP includes over 50 technology accelerators, frameworks, and tools designed to fast-track implementation. We have strong alliances with major AI hyperscalers including Microsoft, Google, and AWS, and are recognized globally as a leader in Data and AI services.

Our AI practice is anchored by Orbitae®, SDG’s agentic AI brand, and INSIGHTGEN®, together with 50+ accelerators and industry agent frameworks that shorten the distance between fragmented enterprise data and deployed, governed AI agents — in the cloud, on-premises, or across hybrid and sovereign environments.

If you’re seeking to join a team fully specialized in Data, Analytics, and AI, we can’t wait to meet you.

Position Overview

As a Databricks Data Engineering Consultant, you will play an important role in designing, developing, and delivering modern data solutions for SDG Group’s clients.

Working as part of a consulting delivery team, you will partner with both technical and business stakeholders to understand data requirements and build reliable, scalable solutions that support analytics, reporting, AI, and broader data-driven initiatives.

This opportunity requires hands-on Databricks experience, with particular emphasis on Databricks SQL and SQL-based data engineering workloads. You will work with complex enterprise data environments, developing and optimizing transformations, validating data, supporting pipelines, and helping ensure that data is structured and delivered effectively for downstream business and analytical use cases.

This role is well suited for someone with a strong foundation in data engineering who enjoys hands-on technical delivery, solving complex problems, working directly with clients, and continuing to expand their expertise across modern cloud and data platforms.

Key Responsibilities

• Databricks Development: Design, build, test, and maintain scalable data solutions within Databricks, with a strong focus on SQL-based transformation and data processing workloads.

• SQL Development: Write, troubleshoot, and optimize complex SQL queries used for data transformation, validation, reconciliation, and analytical workloads.

• Data Engineering & Development: Build and maintain scalable data pipelines and processing solutions that support client analytics and business requirements.

• Data Integration: Integrate data from a variety of structured and unstructured sources using modern ETL/ELT methodologies and data integration tools.

• Data Modeling & Warehousing: Develop and maintain data models and warehouse structures that support reporting, analytics, and downstream applications.

• Cloud Data Solutions: Contribute to the implementation and modernization of cloud-based data platforms using Databricks and related cloud technologies.

• Technical Delivery: Transform, validate, and manage enterprise data while following development, testing, documentation, and deployment best practices.

• Data Quality & Performance: Identify, troubleshoot, and resolve data quality issues while improving reliability, scalability, and query performance.

• Client Collaboration: Work directly with clients and internal stakeholders to gather requirements, understand business challenges, communicate technical concepts, and deliver solutions aligned with client needs.

• Team Collaboration: Work closely with data engineers, analysts, architects, data scientists, and business stakeholders throughout the project lifecycle.

• Continuous Learning: Stay current with evolving Databricks capabilities, data engineering methodologies, and industry best practices while contributing knowledge and ideas to the broader SDG community.

Qualifications

• Professional Experience: Approximately 2–5+ years of experience in data engineering, data warehousing, data integration, analytics engineering, or a related technical discipline. Experience within consulting or another client-facing environment is a plus.

• Databricks Experience: Hands-on professional experience working within Databricks is required, with demonstrated use of the platform for production data engineering and analytical workloads.

• Databricks SQL: Strong experience using Databricks SQL, including writing, troubleshooting, and optimizing queries used for data transformation, validation, reconciliation, and downstream analytics.

• SQL & Programming: Strong SQL skills, including joins, CTEs, window functions, aggregations, and complex transformations. Working proficiency in Python or another relevant programming or scripting language.

• Data Engineering Fundamentals: Strong understanding of data pipelines, ETL/ELT processes, data integration, data warehousing, and data transformation concepts.

• Data Modeling: Understanding of dimensional modeling and modern data modeling concepts used within analytical environments.

• Cloud & Data Platforms: Experience working with modern cloud environments and enterprise-scale data platforms.

• Data Integration Tools: Experience with modern data engineering, orchestration, or transformation tools. Experience with Matillion and/or dbt is highly preferred.

• Data Quality: Experience validating, reconciling, and troubleshooting data across multiple source and target systems.

• Problem Solving: Strong analytical skills with the ability to troubleshoot data issues, understand complex requirements, and translate business needs into technical solutions.

• Consulting & Communication Skills: Strong written and verbal communication skills with the ability to collaborate effectively with clients, technical teams, and business stakeholders.

• Growth Mindset: A proactive, collaborative approach and willingness to continuously learn new tools, technologies, and methodologies.

Preferred Experience

• Databricks SQL Warehouses

• Delta Lake / Delta Tables

• Databricks Workflows and orchestration

• Unity Catalog

• Databricks query and performance optimization

• Matillion

• dbt

• Python / PySpark

• Snowflake

• Experience supporting enterprise-scale data migration or modernization initiatives

• Experience working within pharmaceutical, life sciences, healthcare, or another highly regulated industry

What We Offer

• Stability and Growth: Full-time employment in a growing high-tech industry with a meritocratic culture that supports career advancement and professional development.

• Competitive Compensation: Competitive salary and a 401(k) retirement plan with employer match.

• Comprehensive Benefits: Robust healthcare and benefits package including Medical, Dental, Vision, and Disability coverage.

• Work-Life Balance: 3+ weeks of paid vacation, a flexible holiday schedule, and hybrid working arrangements that balance on-site and remote work.

• Supportive Community: A collaborative environment where you will work alongside experienced professionals, learn from your colleagues, and contribute to meaningful client engagements.

• Focus on Technology: Continuous training and exposure to modern Data, Analytics, and AI technologies, supported by SDG Group’s partnerships with leading technology providers.