Details:
Stefanini Group is hiring!
Stefanini is looking for a Lead Cloud Data Engineer (Data Mesh & AI) for San Francisco, CA/Los Angeles, CA (Onsite)
For quick Apply, please reach out to Prakhar Goel: 248 263 5255/ Prakhar.goel@stefanini.com
W2 candidates Preferred!
We are seeking an experienced Lead Data Engineer to join our team and drive the development of modern data platforms and AI-augmented solutions. This role is critical to supporting ongoing operations of Large Data Warehouse, architecting the Large Data Hub, and preparing for the enterprise data mesh modernization program.
Responsibilities:
Data Engineering & Pipeline Development:
- Develop and maintain end-to-end ETL/ELT pipelines for data ingestion from multiple sources (databases, APIs, streaming), transformation, quality validation, data modeling, and consumption layer optimization using AWS services (Glue, EMR, Lambda, Kinesis, Step Functions)
- Implement data governance policies, metadata management, data lineage tracking, fine-grained access controls, and automated data quality frameworks with validation rules, anomaly detection, and monitoring/alerting mechanisms
Data Mesh Architecture:
- Design and implement Data Mesh architecture on AWS with federated governance, defining domain boundaries, data product specifications, and self-serve infrastructure patterns across Databricks, Starburst, Collibra, and Immuta platforms
- Develop reusable frameworks, accelerators, and self-serve tools enabling domain teams to independently publish, discover, and consume data products while ensuring performance optimization and cost efficiency
DevSecOps & Infrastructure:
- Integrate DevSecOps practices including CI/CD pipelines, infrastructure as code (Terraform/CloudFormation), automated security scanning, compliance validation, and disaster recovery strategies
AI-Augmented Development:
- Design and build agentic AI solutions, RAG pipelines, and intelligent automation using LLMs, orchestration frameworks, and AI-powered development tools (Copilot, Claude Code) to enhance platform capabilities and accelerate SDLC
Technical Leadership & Collaboration:
- Provide L3 technical support and mentor engineering teams on data mesh principles and modern data technologies
- Collaborate with stakeholders to translate business requirements into technical solutions
- Lead architecture design reviews and create comprehensive technical documentation
#LI-PG1
#LI-ONSITE