Senior Data Engineer, BI United Nations Federal Credit Union
New York, NY30+ days ago
- $115,000 Per Year
- Full-time
TechDigital Corporation
- Jersey City, NJ
- 30+ days ago
- Instant Apply
Want to know if you’re a fit?
Upload your resume and let our AI show you.
Upload your resume and let our AI show you.
Skills
- Access Controlunmatched
- Analysis Skillsunmatched
- Apache Sparkunmatched
- Artificial Intelligence (AI)unmatched
- Balance Sheetunmatched
- Banking Servicesunmatched
- Business Intelligenceunmatched
- Cloud Computingunmatched
- Cryptographyunmatched
- Data Analysisunmatched
- Data Managementunmatched
- Data Migrationunmatched
- Data Qualityunmatched
- Datasheetsunmatched
- Financial Operationsunmatched
- Financial Servicesunmatched
- Reconciliationunmatched
- Refactoringunmatched
- Regulatory Complianceunmatched
- Regulatory Reportsunmatched
- Riskunmatched
- SQL (Structured Query Language)unmatched
- Scalable System Developmentunmatched
- Software Engineeringunmatched
- System Migrationunmatched
- Technical Operationsunmatched
- Traceabilityunmatched
- Use Casesunmatched
Description
Role Overview
This role is part of a multi year enterprise initiative to modernize data platforms by migrating from legacy and on prem environments to cloud native, governed, and scalable architectures .
The role focuses on migrating enterprise data workloads to Databricks on strategic cloud platforms , enabling standardized data engineering, analytics, centralized reporting, reconciliation utilities, and AI/ML use cases while adhering to controls, security, resilience, and regulatory compliance .
Key Responsibilities
This role is part of a multi year enterprise initiative to modernize data platforms by migrating from legacy and on prem environments to cloud native, governed, and scalable architectures .
The role focuses on migrating enterprise data workloads to Databricks on strategic cloud platforms , enabling standardized data engineering, analytics, centralized reporting, reconciliation utilities, and AI/ML use cases while adhering to controls, security, resilience, and regulatory compliance .
Key Responsibilities
- Lead and execute migration of legacy data platforms (on prem / non standard tools) to Databricks on cloud under the Olympus program
- Perform application, data, and pipeline refactoring to cloud native Databricks patterns
- Drive migration planning including dependency analysis, sequencing, and cutover strategy
- Support coexistence models and transition from dual run to cloud only execution
- Design and implement Databricks Lakehouse architecture (Bronze / Silver / Gold)
- Build scalable batch and streaming pipelines using PySpark, Spark SQL
- Leverage Delta Lake for reliability, versioning, and performance
- Optimize compute usage and cost in line with enterprise cloud efficiency goals
- Embed data quality, reconciliation, and completeness controls as part of migration
- Ensure migrated workloads meet EDO governance, MCA, and audit requirements
- Maintain lineage, traceability, and explainability across migrated assets
- Support risk critical use cases (Finance, Ops, Recon, Reporting)
- Implement cloud aligned RBAC, identity controls, and secure access patterns
- Enforce data encryption, masking, and classification standards
- Ensure workloads meet operational resilience and recovery expectations
- Partner with cloud platform and security teams for certification and sign off
- Enable downstream BI, regulatory reporting, and MI workloads on Databricks
- Support centralized reporting programs (e.g., ARA, GRU related use cases)
- Prepare data foundations for AI / ML and Agentic workflows post migration
- 8 12+ years in data engineering / platform modernization
- Strong hands on experience with Databricks in large scale enterprises
- Proven experience delivering cloud migration programs (on prem cloud)
- Deep expertise in Apache Spark, PySpark, Spark SQL
- Experience embedding controls, reconciliation, and data quality in migrations
- Experience in regulated environments (banking / financial services preferred)
- Experience with Client Olympus or equivalent enterprise cloud programs
- Knowledge of legacy data platforms and modernization patterns
- Familiarity with Finance, Ops, Recon, or Balance Sheet data domains
- Exposure to MLflow, AI pipelines, or GenAI enablement on cloud
- Strong understanding of run the bank vs change the bank execution
- Strong ownership and execution mindset
- Comfortable operating in large, multi vendor transformation programs
- Ability to engage with Technology, Operations, Risk, and Audit stakeholders
- Disciplined approach to migration risk, controls, and documentation
Numbers & Facts
| Location | Jersey City, NJ |
| Industry | Other/Not Classified |
| Company Size | 100 to 499 employees |
Similar Jobs
- New!
Senior Data Engineer Mitchell Martin
New York, NY5 days ago- $130,000–$170,000 Per Year
- Instant Apply
- New!
Data Engineer ECLARO
Rutherford, NJ1 day ago- Instant Apply
Data Engineer -601/602 Pinnacle Technical Resources
Jersey City,, New Jersey30+ days ago- $65–$70 Per Hour
- Contractor
- New!
Vice President - Senior Data Engineer Barclays
Whippany, NJ6 days ago- $170,000 Per Year
Lead Data Engineer Vaco LLC
New York, NY9 days agoRemote- $165,000–$180,000 Per Year