Agile Programming Methodologies, Application Integration, Application Programming Interface (API), Best Practices, Business Intelligence, Business Solutions, Cloud Architecture, Cloud Computing, Code Reviews, Communication Skills, Continuous Deployment/Delivery, Continuous Integration, Cost Control, Cross-Functional, Data Modeling, Data Processing, Data Quality, DevOps, Dimensional Modeling, Enterprise Applications, Enterprise Architecture, Git, GitHub, Information/Data Security (InfoSec), Leadership, Mentoring, Microservices, Microsoft Windows Azure, Microsoft Windows Mobile, Performance Tuning/Optimization, Production Systems, SQL (Structured Query Language), Safety/Work Safety, Software Development, Software Engineering, Team Player, Technical Leadership, Technical/Engineering Design, Test Plan/Schedule, Theater Production
Lead Databricks Integration Architect
Location: Washington, DC (Hybrid Onsite for PI Planning & Key SAFe Ceremonies)
Job Type: Contract
Duration: July 27, 2026 June 30, 2027
Job Summary
We are seeking an experienced Lead Databricks Integration Architect to join a high-impact enterprise data transformation initiative in Washington, DC. This is a hands-on technical leadership role responsible for designing, building, and governing enterprise-grade integration solutions using the Databricks platform.
The ideal candidate will possess deep expertise in Databricks, Delta Live Tables (DLT), PySpark, Structured Streaming, SQL, and Delta Lake, along with a proven ability to architect scalable cloud data platforms and integrate them with enterprise applications, APIs, and business intelligence solutions. In addition to solution architecture, this role will provide technical leadership through mentoring, code reviews, and the development of reusable engineering frameworks.
The position operates within a SAFe (Scaled Agile Framework) environment and requires participation in key on-site ceremonies, including Program Increment (PI) Planning in Washington, DC.
Required Skills
- 8+ years of experience in Solution Architecture, Data Engineering, or Enterprise Application Architecture.
- Extensive hands-on experience with Databricks platform development.
- Strong expertise in PySpark, SQL, and Delta Lake.
- Advanced experience designing and implementing Delta Live Tables (DLT) pipelines.
- Deep knowledge of Structured Streaming and real-time data processing.
- Strong understanding of Medallion Architecture (Bronze, Silver, Gold).
- Experience designing relational and dimensional data models for enterprise analytics.
- Expertise in application integration using APIs, microservices, and enterprise data platforms.
- Strong experience with CI/CD implementation using Git, Azure DevOps, GitHub Actions, or Terraform.
- Experience implementing Infrastructure as Code (IaC).
- Excellent leadership, mentoring, and stakeholder communication skills.
- Experience working within SAFe Agile environments.
Preferred Skills
- Experience building reusable Databricks platform libraries and engineering frameworks.
- Strong understanding of cloud cost optimization (FinOps).
- Experience designing enterprise data governance and lineage solutions.
- Knowledge of enterprise analytics and business intelligence integrations.
Key Responsibilities
- Design and implement scalable integration architectures connecting Databricks with enterprise applications, APIs, microservices, and business intelligence platforms.
- Develop reusable engineering libraries, frameworks, and abstraction layers to accelerate application development.
- Build and optimize Delta Live Tables (DLT) pipelines and Structured Streaming solutions for real-time data processing.
- Architect enterprise data quality frameworks using DLT Expectations, automated validation, and data lineage tracking.
- Lead architecture discussions, conduct technical design reviews, and mentor engineering teams on best practices.
- Optimize Databricks workloads using Photon, Z-Ordering, Liquid Partitioning, and other performance tuning techniques while improving cloud cost efficiency.
- Design and govern enterprise CI/CD deployment strategies across Development, Testing, Staging, and Production environments.
- Collaborate with Cloud Infrastructure, Platform Engineering, Data Engineering, and Business stakeholders to deliver scalable and secure data integration solutions.
- Ensure architecture aligns with enterprise standards, scalability, security, and operational excellence.
Interview & Work Environment
- Hybrid work model based in Washington, DC.
- Onsite attendance required for Program Increment (PI) Planning and other key SAFe ceremonies.
- Collaborative Agile environment with cross-functional engineering teams.