Access Control, Agile Programming Methodologies, Apache Spark, Application Programming Interface (API), Artificial Intelligence (AI), Automation, Best Practices, Cloud Architecture, Cloud Computing, Code Reviews, Communication Skills, Computer Science, Continuous Deployment/Delivery, Continuous Integration, Cross-Functional, Data Management, Ecosystems, Enterprise Data Integration, Enterprise Protection, Hardware Virtualization, Information Technology & Information Systems, Leadership, Machine Learning, Mentoring, Metadata, Python Programming/Scripting Language, Quality Engineering, SAP, SQL (Structured Query Language), Software Engineering, Standards Development, Strategic Planning, Technical Delivery, Technical Leadership, Technical Strategy, Virtualization
Senior Data & AI Platform Engineer
Location: Washington, DC (Onsite)
Employment Type: Contract
Contract Duration: July 20, 2026 June 30, 2027
Number of Openings: 1
Position Overview
We are seeking a highly skilled Senior Data & AI Platform Engineer to design, build, and scale a modern enterprise data platform supporting advanced analytics and AI initiatives. This role is ideal for a hands-on technical leader with expertise in Databricks, Apache Spark, Python, SQL, and MLOps, who can architect cloud-native data solutions and mentor engineering teams.
The successful candidate will lead the development of a unified, self-service data ecosystem, integrating modern lakehouse technologies with enterprise systems while enabling AI-driven innovation through scalable data engineering and platform automation.
Key Responsibilities
Data & Platform Architecture
- Design and evolve a scalable Data & AI platform using Databricks Lakehouse Architecture.
- Integrate enterprise data sources, including SAP platforms and data virtualization technologies, into a governed and self-service ecosystem.
- Define architecture standards that support scalability, security, and enterprise-wide analytics.
Data Engineering & Solution Development
- Lead the development of robust data pipelines and transformation frameworks using Apache Spark, Python, and SQL.
- Build and maintain feature stores, reusable data products, and API-driven integrations.
- Develop high-performance solutions that support reporting, analytics, and AI workloads.
MLOps & AI Enablement
- Design and implement production-ready MLOps pipelines with automated deployment, monitoring, model versioning, and CI/CD practices.
- Enable efficient machine learning lifecycle management and operationalization of AI models.
- Support the creation and maintenance of internal AI platform services and reusable components.
Platform Governance & Automation
- Implement governance-as-code practices by embedding security, compliance, and quality controls into engineering workflows.
- Establish standards for data ingestion, transformation, metadata management, and operational monitoring.
- Promote automation and best practices across the data engineering lifecycle.
Technical Leadership & Mentorship
- Provide technical leadership through architecture reviews, code reviews, and engineering best practices.
- Mentor junior and mid-level engineers to foster technical excellence and professional growth.
- Collaborate with cross-functional teams to solve complex technical challenges.
Stakeholder Collaboration
- Partner with architects, product owners, and governance teams to align technical delivery with enterprise data strategy.
- Communicate technical concepts effectively to both engineering and business stakeholders.
- Contribute to roadmap planning and strategic technology initiatives.
Required Qualifications
- Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related field (or equivalent professional experience).
- 4 6+ years of experience in Data Engineering, Software Engineering, Data Platform Engineering, or Data Architecture.
- Advanced proficiency in Python and SQL.
- Extensive hands-on experience with Apache Spark and modern Lakehouse architectures, preferably Databricks.
- Proven expertise in designing and implementing scalable data pipelines and enterprise data platforms.
- Experience building and supporting MLOps pipelines, feature stores, semantic layers, or similar AI platform services.
- Strong understanding of API-first architectures, event-driven systems, and secure service-to-service communication.
- Experience implementing Role-Based Access Control (RBAC) and enterprise security best practices.
- Demonstrated ability to lead technical initiatives within Agile development environments.
Preferred Skills
- Experience integrating enterprise data platforms with SAP technologies such as SAP Datasphere or SAP S/4HANA.
- Familiarity with data virtualization platforms such as Denodo.
- Knowledge of CI/CD automation for data engineering and machine learning workflows.
- Experience building cloud-native analytics platforms and reusable data services.
Key Skills
- Databricks
- Apache Spark
- Python
- SQL
- Lakehouse Architecture
- Data Engineering
- MLOps
- Machine Learning Platforms
- Feature Stores
- API Development
- Event-Driven Architecture
- CI/CD
- RBAC Security
- Data Governance
- Data Platform Engineering
- Agile Delivery
- Technical Leadership
- Enterprise Analytics