Senior Infrastructure Architect Azure & Azure Databricks W2 position
Location: Remote
Experience: 8 years
Work schedule: Day shift
Engagement: Senior Consultant
Role Summary
We're looking for a Senior Infrastructure Architect with strong experience designing and supporting Azure and Azure Databricks platforms. In this role, you'll build secure, scalable analytics environments, support application teams as they onboard and migrate to Databricks, and help improve data access and platform operations for enterprise clients.
You'll work closely with cross-functional teams in an Agile environment. The role focuses on platform architecture, provisioning, configuration, and operational support.
Key Responsibilities
- Design and implement Databricks Delta Sharing across the organization and its affiliates.
- Deploy and configure Azure Databricks workspaces.
- Onboard applications to Databricks and support application teams through migration.
- Build and configure Unity Catalog, including troubleshooting workspace access issues.
- Design, develop, and optimize data pipelines and workflows using PySpark, ADLS, and Azure Data Factory.
- Create and configure Azure resources, access controls, and role-based access control (RBAC).
- Provision, secure, and support Azure services used by application teams.
- Design and deploy Platform as a Service (PaaS) solutions using Azure services.
- Troubleshoot Azure resource issues across IaaS and PaaS services.
- Apply Azure high availability and disaster recovery patterns.
- Work with Azure networking, Azure Policy, Linux, and scripting tools such as Bash and Python.
- Use ARM templates to provision and manage Azure infrastructure.
- Collaborate with cross-functional teams using Agile methodologies.
Required Qualifications
- Approximately 7 years of experience in infrastructure, cloud, or data platform engineering.
- Hands-on experience writing Spark jobs using PySpark.
- Experience integrating data platforms with Azure Data Lake Storage (ADLS).
- Experience building and managing Azure Data Factory (ADF) pipelines.
- Experience with Azure Databricks workspaces, Unity Catalog, and Delta Sharing.
- Strong knowledge of Azure resource provisioning, security, and RBAC.
- Experience supporting Azure IaaS and PaaS services in production environments.
- Familiarity with Azure networking, Azure Policy, Linux, Python or Bash scripting, and ARM templates.
- Understanding of Azure high availability and disaster recovery approaches.
- Experience working with Agile teams.
Preferred Experience
Experience provisioning, securing, or supporting application teams using Azure AI and cognitive services, including:
- Azure Bot Service and chatbot solutions
- Azure AI Language (formerly QnA Maker and Text Analytics)
- Azure AI Search
- Azure Translator
- Azure Speech services
- LUIS (Language Understanding) and related services