Solution Architect - Data Engineering (Azure Databricks)

Solution Architect - Data Engineering (Azure Databricks)
Location: Remote
Rate: $75/hr on C2C
Indent: PSL303558-141-5

Role Summary
We are seeking a Solution Architect (Contractor) to design and drive enterprise-scale data engineering architectures on Azure Databricks and cloud platforms. This role will define end-to-end data solutions, architecture patterns, and engineering standards to support analytics, AI/ML, and operational use cases.
The ideal candidate will combine deep technical expertise with strong business alignment, enabling scalable, secure, and high-performing data platforms.
Solution architecture is critical to ensuring standardized, scalable, and governed data ecosystems across domains and use cases[NextGen Da...quirements | Meeting]

Key Responsibilities
1. Architecture & Solution Design

• Define and implement end-to-end data architecture (ingestion → transformation → consumption)
• Design scalable Lakehouse architectures (Databricks, Delta Lake)
• Develop reference architectures, reusable frameworks, and design patterns
• Align architecture with performance, scalability, security, and cost optimization goals

2. Data Modeling & Domain Architecture

• Define canonical data models, dimensional schemas, and semantic layers
• Architect enterprise data across domains:
  • Claims, Member, Provider, Eligibility
• Ensure data consistency, interoperability, and reuse across teams

3. Azure Databricks & Cloud Enablement

• Architect solutions using:
  • Azure Databricks (Mandatory)
  • Azure Data Services (ADF, ADLS, Synapse)
• Guide engineering teams on:
  • Spark optimization, Delta Lake design, and data pipeline best practices
• Enable multi-cloud integrations when required

4. Governance, Security & Compliance

• Define and enforce:
  • Data governance frameworks (lineage, quality, metadata)
  • Access control, RBAC, and data security policies
• Ensure compliance with:
  • HIPAA, HITRUST, and enterprise regulatory standards
• Establish data quality SLAs and auditability mechanisms

5. Engineering & Platform Alignment

• Work closely with:
  • Data Engineering teams
  • Platform Engineering teams
  • AI/ML Engineering teams
• Provide architectural oversight for pipelines, workflows, and platform capabilities
• Drive standardization across CI/CD, DataOps, and MLOps practices

6. Stakeholder & Business Engagement

• Collaborate with business, product, and analytics teams to:
  • Translate requirements into scalable technical solutions
• Lead architecture reviews, design approvals, and governance forums (ARB)
• Communicate architecture decisions and trade-offs to leadership

Required Skills

• Strong experience in:
  • Azure Databricks (Mandatory) – PySpark, Delta Lake
  • Cloud Architecture (Azure preferred)
• Deep expertise in:
  • Data Architecture & Data Modeling (OLTP, OLAP, Dimensional)
  • Lakehouse / Data Warehouse design patterns
• Hands-on experience with:
  • ETL/ELT pipelines and orchestration frameworks
  • CI/CD and DevOps practices (Azure DevOps, Git)