Job Title: Data Engineer Python, dbt & Oracle
Location: Jersey City, NJ Hybrid, 3 days onsite
Worksite: 185 Hudson Street, Suite 1150, Jersey City, NJ 07311
Work Authorization: U.S. Citizens and Green Card holders only
Candidate Requirement: Local candidates only
Interview Process: First-round video interview followed by a two-hour onsite final interview
Position Overview
Our Client is seeking an experienced Data Engineer to support a major data infrastructure modernization initiative. This project involves assessing and modernizing approximately 700 800 legacy Perl scripts and RunMyJobs/PL/SQL scheduling workflows.
The selected candidate will help migrate these legacy processes to a modular, observable, and testable data platform using Python, dbt, Apache Airflow, and Oracle 19c.
Key Responsibilities
- Assess, redesign, and migrate legacy Perl scripts, RunMyJobs workflows, and PL/SQL scheduling logic to Python scripts and Apache Airflow DAGs.
- Design, build, and deploy production-grade Airflow DAGs for complex, interdependent enterprise batch workflows.
- Develop custom Airflow operators and sensors for Oracle 19c, REST APIs, SFTP/FTP transfers, and external triggers.
- Build and maintain data transformation workflows using dbt.
- Write and optimize complex Oracle SQL and PL/SQL queries.
- Improve the performance and reliability of data pipelines operating across on-premises systems.
- Implement error handling, retry processes, SLA alerts, execution logs, audit trails, and data-lineage monitoring.
- Troubleshoot failed or partially completed jobs across application logs, databases, and Linux/Unix filesystems.
- Work closely with DBAs and application teams to document job dependencies, identify critical processing paths, and improve data quality and recoverability.
- Support parameter-driven automation and reusable pipeline-development standards.
Required Qualifications
- Strong hands-on experience with dbt and Oracle 19c. Both are mandatory.
- At least five years of Python development experience, particularly in automation, scripting, and enterprise ETL pipelines.
- At least two years of experience developing and deploying production-grade Apache Airflow DAGs, custom operators, and sensors.
- Strong experience with Oracle SQL, PL/SQL development, complex queries, and performance tuning.
- Advanced knowledge of Linux/Unix environments and shell-based processing.
- Experience modernizing or replacing enterprise job schedulers such as RunMyJobs or AutoSys.
- Experience developing REST API integrations and SFTP/FTP file-transfer processes.
- Strong understanding of batch processing, job dependencies, failure recovery, and operational monitoring.
- Proven ability to troubleshoot issues across application logs, databases, filesystems, and data pipelines.
Preferred Qualifications
- Experience modernizing or replacing legacy Perl applications or scripts.
- Previous experience in banking, investment management, or financial services.
- Familiarity with financial data such as holdings, transactions, net asset values, and tax lots.
- Experience with Git or Bitbucket, Jenkins, and CI/CD-based deployment processes.
- Knowledge of operational risk, data governance, audit controls, and data lineage within financial systems.