Data Engineer (Hadoop/Scala/Databricks)

Job title: Data Engineer Azure Synapse & Microsoft Fabric

Location: Atlanta, GA
Experience: 8+ Years

Job Summary

We are seeking a highly skilled Data Engineer with strong expertise in Azure Synapse Analytics, Apache Spark, and modern cloud-based data platforms. The ideal candidate will have hands-on experience designing and implementing scalable data solutions, building robust data pipelines, and leveraging Data Lake architectures to support enterprise analytics and reporting initiatives.

Key Responsibilities

• Design, develop, and optimize scalable data pipelines using Azure-based technologies.
• Build and maintain enterprise data solutions using Azure Synapse Analytics.
• Develop data transformation and processing workflows using PySpark and Spark SQL.
• Design and implement Data Lake solutions following Medallion Architecture (Bronze, Silver, Gold layers).
• Work with structured and unstructured data from multiple sources to support business analytics needs.
• Optimize Spark workloads and troubleshoot performance issues in distributed data processing environments.
• Collaborate with architects, analysts, developers, and business stakeholders to deliver high-quality data solutions.
• Support data quality, governance, and reliability initiatives across the data platform.
• Contribute to the adoption and implementation of Microsoft Fabric capabilities where applicable.

Required Qualifications

• 8+ years of experience in Data Engineering and Data Platform development.
• Strong hands-on experience with Azure Synapse Analytics.
• Solid understanding of Microsoft Fabric and its data engineering capabilities.
• Deep knowledge of Apache Spark architecture and distributed computing concepts.
• Extensive experience with PySpark and Spark SQL.
• Strong understanding of Data Lake concepts and modern data architecture patterns.
• Experience implementing Medallion Architecture and large-scale data processing solutions.
• Strong SQL development and performance tuning skills.
• Excellent analytical, troubleshooting, and problem-solving abilities.
• Strong verbal and written communication skills.

Preferred Qualifications

• Experience with Azure Data Lake Storage (ADLS), Azure Data Factory, and other Azure data services.
• Exposure to data warehousing, ETL/ELT frameworks, and cloud-native analytics solutions.
• Experience working in Agile/Scrum environments.
• Knowledge of data governance, security, and performance optimization best practices.