Lead DATA ENGINEER - Cloud Migration

Role Overview
This role is part of a multiyear enterprise initiative to modernize data platforms by migrating from legacy and onprem environments to cloudnative, governed, and scalable architectures .
The role focuses on migrating enterprise data workloads to Databricks on strategic cloud platforms , enabling standardized data engineering, analytics, centralized reporting, reconciliation utilities, and AI/ML use caseswhile adhering to controls, security, resilience, and regulatory compliance .

Key Responsibilities

• Lead and execute migration of legacy data platforms (onprem / nonstandard tools) to Databricks on cloud under the Olympus program
• Perform application, data, and pipeline refactoring to cloudnative Databricks patterns
• Drive migration planning including dependency analysis, sequencing, and cutover strategy
• Support coexistence models and transition from dualrun to cloudonly execution

Databricks Lakehouse Engineering

• Design and implement Databricks Lakehouse architecture (Bronze / Silver / Gold)
• Build scalable batch and streaming pipelines using PySpark, Spark SQL
• Leverage Delta Lake for reliability, versioning, and performance
• Optimize compute usage and cost in line with enterprise cloud efficiency goals

Enterprise Data Controls & Governance

• Embed data quality, reconciliation, and completeness controls as part of migration
• Ensure migrated workloads meet EDO governance, MCA, and audit requirements
• Maintain lineage, traceability, and explainability across migrated assets
• Support riskcritical use cases (Finance, Ops, Recon, Reporting)

Cloud Security & Resilience

• Implement cloudaligned RBAC, identity controls, and secure access patterns
• Enforce data encryption, masking, and classification standards
• Ensure workloads meet operational resilience and recovery expectations
• Partner with cloud platform and security teams for certification and signoff

Reporting, Analytics & AI Enablement

• Enable downstream BI, regulatory reporting, and MI workloads on Databricks
• Support centralized reporting programs (e.g., ARA, GRUrelated use cases)
• Prepare data foundations for AI / ML and Agentic workflows postmigration

Required Qualifications

• 812+ years in data engineering / platform modernization
• Strong handson experience with Databricks in largescale enterprises
• Proven experience delivering cloud migration programs (onprem cloud)
• Deep expertise in Apache Spark, PySpark, Spark SQL
• Experience embedding controls, reconciliation, and data quality in migrations
• Experience in regulated environments (banking / financial services preferred)

Preferred Qualifications

• Experience with Client Olympus or equivalent enterprise cloud programs
• Knowledge of legacy data platforms and modernization patterns
• Familiarity with Finance, Ops, Recon, or BalanceSheet data domains
• Exposure to MLflow, AI pipelines, or GenAI enablement on cloud
• Strong understanding of runthebank vs changethebank execution

Behavioral & Delivery Expectations

• Strong ownership and execution mindset
• Comfortable operating in large, multivendor transformation programs
• Ability to engage with Technology, Operations, Risk, and Audit stakeholders
• Disciplined approach to migration risk, controls, and documentation