Databricks Platform Engineer Position Summary We are seeking a skilled Databricks Platform Engineer to design, implement, and maintain our enterprise Databricks environment. This role will focus on enabling scalable data processing, optimizing performance, and ensuring security and compliance across the platform. The ideal candidate will have strong experience with cloud technologies, data engineering principles, and automation practices.
Databricks Platform Engineer Key Responsibilities
Platform Administration & Optimization
Deploy, configure, and manage Databricks workspaces, clusters, and jobs.
Monitor platform performance and optimize resource utilization.
Implement best practices for cost management and scalability.
Knowledge of Unity Catalog Implementation
Security & Compliance
Configure identity and access management (IAM) for Databricks.
Ensure compliance with organizational security standards and data governance policies.
Manage secrets, tokens, and secure data access.
Integration & Automation
Integrate Databricks with cloud storage, data lakes, and enterprise systems.
Automate cluster provisioning, job scheduling, and CI/CD pipelines.
Develop reusable scripts and templates for infrastructure as code (IaC).
Collaboration & Support
Partner with Data Architecture, Enterprise Architecture, and Enterprise Infrastructure
Partner with data engineers, analysts, and data scientists to enable efficient workflows.
Provide technical support and troubleshooting for Databricks-related issues.
Document platform architecture, processes, and operational guidelines.