Data Engineer with Databricks

Request ID: 107834-1
Title: Data Engineer with Databricks
Locations: Tampa, FL
Duration: 6 Months
Pay Range: $44 - $47/Hour on W2/C2C (All inclusive)

We are seeking an experienced Data Engineer with 4 6 years of experience and strong hands-on expertise in Databricks and Apache Spark. The ideal candidate will design, develop, and optimize scalable data pipelines to support analytics and data-driven business initiatives.

The candidate will work closely with data scientists, analysts, business stakeholders, and cross-functional teams to ensure data is reliable, secure, high-quality, and readily available.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using Databricks and Apache Spark.
• Build and optimize batch and real-time/streaming data pipelines for large volumes of structured and unstructured data.
• Develop data engineering solutions using Databricks, Spark, and related data technologies.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.
• Monitor, troubleshoot, and optimize data processing workflows to ensure reliability and performance.
• Optimize Databricks clusters for performance, scalability, and cost efficiency.
• Implement data quality, governance, security, and compliance standards across data pipelines.
• Perform data validation and troubleshooting to identify and resolve pipeline issues.
• Maintain technical documentation related to data architecture, data pipelines, workflows, and processes.
• Support continuous improvement of data engineering processes and platform capabilities.
• Explore and integrate AI/Agentic AI capabilities into data engineering workflows where applicable.

Required Skills

• 4 6 years of experience in Data Engineering.
• Strong hands-on experience with Databricks.
• Strong experience with Apache Spark.
• Experience developing ETL/ELT pipelines.
• Experience with both batch and streaming data processing.
• Experience working with large volumes of structured and unstructured data.
• Strong understanding of data engineering and data processing concepts.
• Experience with data pipeline monitoring, troubleshooting, and performance optimization.
• Knowledge of data governance, security, and compliance.
• Experience optimizing Databricks clusters for cost and performance.
• Strong problem-solving and analytical skills.
• Good communication and collaboration skills.

Preferred Skills

• Experience with AI Agents / Agentic AI.
• Experience integrating AI capabilities into data platforms or data workflows.
• Knowledge of cloud data platforms such as AWS, Azure, or GCP.
• Experience with streaming technologies such as Kafka.
• Knowledge of data lake/lakehouse architecture.