Job Description
Azure Databricks Administration & Management:- Deploy, configure, and manage Azure Databricks workspaces in a scalable, cost-efficient, and secure manner.
- Administer clusters, jobs, notebooks, and workflows, ensuring high availability and performance.
- Monitor and optimize compute resource utilization and autoscaling strategies to improve cost efficiency.
- Manage Databricks Runtime versions, libraries, and dependencies across environments.
Security & Compliance:- Implement and manage Unity Catalog for fine-grained access control and data governance.
- Enforce Role-Based Access Control (RBAC) and integrate Databricks with Azure Active Directory (AAD).
- Ensure compliance with SOC 2, HIPAA, GDPR, and internal security standards.
- Set up audit logging, monitoring, and alerting for security and operational insights.
Performance Optimization & Troubleshooting:- Tune Apache Spark workloads to improve query performance and resource efficiency.
- Analyze and troubleshoot performance bottlenecks in ETL and ML workloads.
- Optimize Delta Lake storage, caching, and indexing strategies for better query execution.
Automation & Infrastructure as Code (IaC):- Automate Databricks workspace deployment using Terraform, ARM Templates, or Databricks REST API.
- Develop and maintain CI/CD pipelines for Databricks job deployment and configuration management.
- Implement monitoring solutions using Azure Monitor, Prometheus, or Grafana.
Collaboration & Integration:- Work closely with data engineers, data scientists, and DevOps teams to support data pipelines and analytics workloads.
- Integrate Databricks with Azure Data Lake, Azure Synapse Analytics, and Snowflake.
- Provide technical guidance and best practices for efficient Spark job execution and cost optimization.
Required Skills & Experience:- 5+ years of experience in Azure Databricks administration and performance optimization.
- Expertise in Apache Spark, PySpark, SQL, and Scala.
- Hands-on experience with Databricks Unity Catalog, Delta Lake, and MLflow.
- Strong knowledge of Azure cloud services (Azure Data Lake, Azure Synapse, Azure Key Vault, etc.).
- Experience in Infrastructure as Code (Terraform, Bicep, or ARM Templates).
- Proficiency in CI/CD pipeline automation using Azure DevOps, GitHub Actions, or Jenkins.
- Strong understanding of network security, identity management (AAD), and encryption best practices.
- Excellent problem-solving skills and ability to troubleshoot complex Databricks workloads.
- Strong communication and documentation skills.
Preferred Qualifications:- Databricks Certified Associate or Professional certification.
- Experience with Azure Kubernetes Service (AKS) and serverless computing.
- Familiarity with Kafka, Apache Airflow, and event-driven architectures.
- Knowledge of Python, PowerShell, or Bash scripting for automation
Numbers & Facts
| Location | Cincinnati, OH |
| Industry | Other/Not Classified |
| Company Size | 100 to 499 employees |
Skills
ARM (Advanced RISC Machine)unmatched
Access Controlunmatched
Administrative Managementunmatched
Apache Kafkaunmatched
Apache Sparkunmatched
Application Programming Interface (API)unmatched
Automationunmatched
Autoscalingunmatched
Bash Scriptingunmatched
Best Practicesunmatched
Cachingunmatched
Cisco Unityunmatched
Cloud Computingunmatched
Communication Skillsunmatched
Configuration Managementunmatched
Continuous Deployment/Deliveryunmatched
Continuous Integrationunmatched
Cost Controlunmatched
Cryptographyunmatched
Data Analysisunmatched
Data Lakeunmatched
Data Managementunmatched
Data Scienceunmatched
Database Extract Transform and Load (ETL)unmatched
DevOpsunmatched
Documentationunmatched
Energy Efficiencyunmatched
GitHubunmatched
HIPAA (Health Insurance Portability and Accountability Act)unmatched
High Availabilityunmatched
Identify Issuesunmatched
Identity Data Managementunmatched
Jenkinsunmatched
Maintain Complianceunmatched
Microsoft Active Directoryunmatched
Microsoft Windows Azureunmatched
Network Securityunmatched
Performance Analysisunmatched
Performance Managementunmatched
Performance Tuning/Optimizationunmatched
Problem Solving Skillsunmatched
Python Programming/Scripting Languageunmatched
REST (Representational State Transfer)unmatched
Resource Utilizationunmatched
SQL (Structured Query Language)unmatched
Scala Programming Languageunmatched
Scripting (Scripting Languages)unmatched
Snowflake Schemaunmatched
Technical Leadershipunmatched
Windows PowerShellunmatched
Level up your application
Browse dozens of recruiter approved resume templates, layouts and formats. Choose your favorite and make it your own in minutes.
Free resume templatesImprove your existing resume or start from scratch and create a standout, ATS-friendly resume. Add job-specific content, download and apply.
Free resume builder