Job Description - Data Migration Development Lead (AWS)
Role Summary : We are seeking an experienced Data Migration Development Lead to lead the design, development, and execution of large-scale data migration solutions using the AWS technology stack.
The ideal candidate will have strong hands-on expertise in data engineering, ETL/ELT development, AWS data services, data quality, reconciliation, and migration cutover. This role will provide technical leadership to the migration development team and work closely with architects, application teams, business stakeholders, and production support teams to ensure accurate, secure, scalable, and reliable movement of data from legacy platforms to target systems.
Key Responsibilities
Data Migration Architecture & Design
* Lead the technical design and implementation of end-to-end data migration solutions.
* Translate migration requirements and source-to-target mappings into scalable technical solutions.
* Define migration patterns for extract, transform, cleanse, validate, load, reconcile, and archive processes.
* Partner with solution and data architects to define migration architecture, data flows, integration patterns, and AWS service selection.
* Design solutions capable of supporting high-volume and complex enterprise data migrations.
AWS Data Engineering
* Design and develop migration pipelines using AWS services such as:
* AWS Glue
* Amazon S3
* AWS Lambda
* AWS Step Functions
* Amazon Redshift
* Amazon RDS/Aurora
* AWS Database Migration Service (DMS)
* Amazon EMR, where appropriate
* Amazon CloudWatch
* AWS Secrets Manager
* AWS IAM
* Build reusable and parameterized ETL/ELT frameworks rather than one-off migration scripts.
* Implement orchestration, restartability, exception handling, logging, monitoring, and operational controls.
* Optimize pipelines for performance, scalability, reliability, and AWS cost efficiency.
Migration Development
* Lead development of extraction, transformation, cleansing, enrichment, and loading processes.
* Implement complex business transformation and data conversion rules.
* Develop incremental/delta migration capabilities and Change Data Capture patterns where required.
* Handle large-volume batch processing and parallelization.
* Design migration processes that support multiple mock conversions and production cutovers.
* Develop automated reconciliation between source and target systems.
Data Quality & Reconciliation
* Establish technical controls to validate:
* Record counts
* Financial/control totals
* Referential integrity
* Mandatory attributes
* Duplicate records
* Transformation accuracy
* Source-to-target completeness
* Build automated data-quality and reconciliation reporting.
* Identify root causes for migration discrepancies and drive remediation.
* Work with business and data owners to establish acceptance criteria and migration sign-off.
Technical Leadership
* Lead and mentor a team of data migration developers.
* Perform design and code reviews and enforce engineering standards.
* Break migration requirements into development deliverables and assign work across the team.
* Estimate development effort and support sprint/release planning.
* Track technical dependencies, risks, defects, and blockers.
* Provide hands-on technical support for complex development and production issues.
* Establish coding standards and reusable migration frameworks across workstreams.
Testing & Cutover
* Support multiple migration cycles including:
* Development testing
* System Integration Testing
* User Acceptance Testing
* Mock conversions
* Dress rehearsals
* Production migration
* Develop migration runbooks and detailed execution procedures.
* Define restart/recovery procedures for failed migration jobs.
* Participate in production cutover planning and execution.
* Support post-migration validation, reconciliation, defect resolution, and stabilization.
DevOps & Automation
* Implement CI/CD practices for migration code and AWS infrastructure.
* Support Infrastructure as Code using Terraform, AWS CloudFormation, or AWS CDK.
* Integrate migration pipelines with enterprise deployment processes.
* Implement automated testing and deployment controls.
* Establish CloudWatch monitoring, alerts, dashboards, and operational logging.
Required Qualifications
* 8+ years of experience in software development, data engineering, ETL, or data migration.
* 3+ years of experience leading technical development teams or major data migration workstreams.
* Strong hands-on experience with AWS data and integration services.
* Advanced experience with AWS Glue, S3, Lambda, Step Functions, DMS, CloudWatch, and IAM.
* Strong development skills in Python and SQL.
* Experience with PySpark/Spark for large-volume data transformation.
* Strong understanding of ETL/ELT architecture and data migration patterns.
* Experience migrating data from relational databases, legacy systems, APIs, files, or enterprise applications.
* Strong knowledge of relational database concepts, data modeling, and SQL performance optimization.
* Experience implementing automated data-quality and reconciliation frameworks.
* Experience supporting large-scale production cutovers.
* Strong troubleshooting and root-cause-analysis skills.
* Experience working within Agile/Scrum delivery models.
Preferred Qualifications
* Experience with enterprise-scale transformation or modernization programs.
* Experience migrating data between legacy/on-premise platforms and cloud-based applications.
* Experience with Amazon Redshift, Aurora, RDS, EMR, Athena, and Lake Formation.
* Experience with CDC and near-real-time migration patterns.
* Experience with Kafka or other event-streaming technologies.
* Experience integrating AWS migration platforms with SaaS or enterprise applications such as Salesforce.
* Experience with Terraform and CI/CD platforms.
* AWS certification such as AWS Certified Data Engineer Associate or AWS Certified Solutions Architect.
Technical Skills
Area Technologies
Cloud AWS
Storage Amazon S3
ETL / Data Integration AWS Glue, PySpark
Database Migration AWS DMS
Development Python, SQL
Serverless AWS Lambda
Orchestration AWS Step Functions
Databases Redshift, Aurora, RDS
Analytics Athena, EMR
Monitoring CloudWatch
Security IAM, KMS, Secrets Manager
DevOps Git, CI/CD, Terraform / CloudFormation / CDK Data Processing Spark / PySpark
Migration Controls Data Quality, Reconciliation, Audit, Restart/Recovery
Key Competencies
The successful candidate should demonstrate:
* Strong hands-on engineering capability, not only project coordination.
* Ability to translate complex source-to-target requirements into maintainable migration code.
* Strong understanding of data quality, reconciliation, lineage, and auditability.
* Ability to troubleshoot performance and data issues across large datasets.
* Ability to lead developers while remaining technically involved.
* Strong ownership of migration outcomes from development through production cutover.
* Ability to communicate technical risks and decisions clearly to architects, program leadership, application teams, and business stakeholders.
Success Measures
Success in this role will be measured by:
* Accuracy and completeness of migrated data.
* Migration execution within required cutover windows.
* Low defect leakage into production.
* Automated reconciliation and auditability.
* Pipeline performance and scalability.
* Effective restart and recovery capabilities.
* Reusability of migration components and frameworks.
* Successful execution of mock migrations and production cutover.
* Technical quality and productivity of the migration development team.