Lead Databricks Integration Architect

I hope you are doing well.

Please find the job details below:

Job Title: Lead Databricks Integration Architect
Location: Washington, DC

Requirement:

Lead Solutions Architect specializing in Databricks and Application Integration on behalf of our client in Washington, D.C.
This practitioner-level leader will create and execute the technical blueprints that link the enterprise Databricks platform directly to downstream operational applications, microservices, business intelligence tools, and external APIs.
Acting as a hands-on mentor and technical authority, this architect will build reusable platform libraries, lead engineering code reviews, and optimize real-time streaming data pipelines.
This position operates under a SAFe (Scaled Agile) framework with a hybrid schedule, requiring on-site attendance for critical ceremonies like PI planning in Washington, D.C.

Key Responsibilities
Hands-on Integration Blueprinting: Design, document, and actively build end-to-end integration patterns utilizing Databricks Delta Live Tables (DLT), structured streaming, and core APIs to connect data with enterprise software.

Framework & Code Library Development:
Partner with Core Platform and Cloud Infrastructure teams to write reusable platform libraries and abstraction layers that speed up development cycles.

Data Quality Mechanics:
Architect and enforce technical data quality frameworks, including automated DLT "Expectations" templates and granular data lineage tracking.

Mentorship & Code Reviews:
Lead architectural review boards, oversee peer code/design reviews, and provide direct technical mentorship to mid-and-senior data engineers.

Performance Tuning (FinOps):
Implement technical optimization strategies (Z-Ordering, Liquid Partitioning, Photon compute) to balance data throughput with cloud processing costs.

CI/CD Lifecycle Governance:
Design and govern multi-tier code deployment strategies (Dev, Test, Stage, Prod) for Databricks assets and automated Infrastructure as Code (IaC).

Required Qualifications

Professional Experience:

• 8+ years of experience in IT solution architecture, application design, or senior-level data engineering, with a history of building integrated cloud data ecosystems.
• Hands-on Databricks & Language Mastery: Authoritative, hands-on engineering experience using the Databricks platform, PySpark, SQL, and core Delta Lake concepts.
• Advanced DLT & Streaming Expertise: Deep technical proficiency building and blueprinting complex Delta Live Tables (DLT) architectures and real-time structured streaming solutions.
• Enterprise Data Modeling: Expert-level understanding of designing relational, dimensional, and Medallion (Bronze/Silver/Gold) configurations optimized for multi-application consumption.
• Automation & CI/CD Architecture: Strong experience designing enterprise deployment strategies and automated infrastructure provisioning using Git, Azure DevOps, GitHub Actions, or Terraform.
• Collaborative Leadership: Exceptional capability to lead technical squads, drive consensus across engineering teams, and clearly explain technical roadmaps to non-technical stakeholders.
• Agile Integration: Ability to thrive in a SAFe (Scaled Agile) environment, including full commitment to iteration planning and on-site PI planning in Washington, D.C.