Lead Data Engineer

Title: Data Engineer - Ab Initio & GCP Modernization

Location: Charlotte, NC

Duration: 103 W, 4 D

Work Engagement: W2

Work Schedule: Onsite

Benefits on offer for this contract position: Health Insurance, Life insurance, 401K and Voluntary Benefits

Summary:

We are seeking an experienced Data Engineer to support enterprise-scale data integration, modernization, and cloud transformation initiatives. This role will be responsible for the development, enhancement, and support of complex data pipelines using Ab Initio, SQL, Python, and PySpark, while contributing to the migration of legacy ETL solutions to modern cloud-based platforms on Google Cloud Platform (GCP).

The ideal candidate will possess strong hands-on experience with Ab Initio development, data warehousing, and large-scale ETL processing, along with production support experience in mission-critical environments. This position requires collaboration with cross-functional teams to design, develop, optimize, and maintain high-performing data solutions that support business and regulatory requirements.

Key Responsibilities:

• Production operations experience: monitoring, SLAs, incident response, root cause analysis, and performance optimization.
• Experience working in hybrid environments (on-prem + cloud) and supporting data migration/modernization initiatives.
• Experience with scheduling/orchestration in Autosys and Airflow-based orchestration (Cloud Composer direction).
• Experience with Git-based workflows, code reviews, and automated testing practices for data pipelines.
• Experience with Harness, Jenkins and uDeploy based CICD environments.
• Practical experience using AI-assisted coding tools in daily development to improve productivity without compromising quality or security.
• Ab Initio development/maintenance experience and/or hands-on migration of Ab Initio graphs to modern Spark/SQL patterns.
• Experience with Dataplex and broader data governance concepts (metadata, classification, stewardship, lineage practices).
• Experience with Informatica Data Quality implementation patterns (profiling, rules, scorecards/metrics, exception workflows).
• Experience designing near real-time patterns (micro-batch/event-driven concepts) and handling late-arriving/out-of-order data.
• Familiarity with GCP operational practices for data workloads (service accounts/IAM basics, job monitoring, quota/cost controls).

Key Requirements:

Applicants must be authorized to work for ANY employer in the U.S. This position is not eligible for visa sponsorship.

• 4+ years of Data Engineering experience or equivalent combination of work experience, education, military experience, and training.

• 4+ years of hands-on Ab Initio development experience, including:

• Complex graph development

• Psets

• Performance tuning

• Production support

• 4+ years of SQL and PL/SQL development experience.

• Proven experience with Oracle, Teradata, and/or BigQuery.

• 4+ years of UNIX Shell Scripting experience.

• 3+ years of Python programming experience.

• 3+ years of hands-on PySpark development experience for distributed data processing.

• Strong understanding of ETL/ELT architectures, data warehousing concepts, and data modeling.

• Experience supporting production environments, monitoring, and troubleshooting large-scale data solutions.