Senior Databricks AI Engineer

Senior Databricks AI Engineer
Atlanta, GA

Location: See Below
Duration: 6 months

Role: Senior Databricks AI Engineer

Position Summary

We are seeking a highly skilled Senior Databricks AI Engineer to design, develop, and deploy scalable data and AI solutions using the Databricks Lakehouse Platform. The ideal candidate will have expertise in Data Engineering, Machine Learning, Generative AI, and cloud technologies, with hands-on experience building enterprise-grade AI applications and data platforms.

Key Responsibilities

Data Engineering

Design and develop scalable data pipelines using Databricks, PySpark, Delta Lake, and Spark SQL.

Build ELT/ETL solutions for structured and unstructured data.

Implement Medallion Architecture (Bronze, Silver, Gold layers).

Optimize Databricks jobs, clusters, and workflows for performance and cost efficiency.

Manage data ingestion from multiple cloud and on-premise sources.

AI & Machine Learning

Develop and deploy Machine Learning models using Databricks ML and Python.

Implement MLOps practices for model training, deployment, monitoring, and governance.

Build predictive analytics solutions and recommendation engines.

Create feature stores and reusable ML pipelines.

Generative AI

Design and develop GenAI applications using:

o Large Language Models (LLMs)

o Retrieval Augmented Generation (RAG)

o Vector Databases

o AI Agents and Multi-Agent Frameworks

Role Descriptions: Design and develop scalable data pipelines using Databricks. Implement machine learning models and MLOps practices. Optimize workflows for performance and cost efficiency. Collaborate with stakeholders to deliver AI-driven solutions. Develop generative AI applications and integrate various AI technologies.
Essential Skills: Experience in healthcare or banking domains is beneficial. Familiarity with AI governance and agentic AI is a plus. Knowledge of MLOps practices is advantageous. Understanding of data modernization programs is helpful. Certification in Databricks or machine learning is preferred.Strong expertise in Databricks and data engineering is essential. Proficiency in machine learning and AI model deployment is critical. Experience with cloud platforms like Azure or AWS is necessary. Familiarity with data pipeline design and optimization is required. Knowledge of generative AI technologies is a must.
Desirable Skills:
Keyword:
Skills: Digital : Machine Learning~Digital : Cloud Foundry~Digital : Databricks~Digital : PySpark~Generative AI
Experience Required: 4-6