Senior Data Engineer

Location: San Jose, CA – Local Candidates Only
Job Type: Contract
Work Arrangement: Onsite – No Remote

Responsibilities

• Lead end-to-end migration, transformation, and loading of high-volume, complex data into PostgreSQL.
• Design, build, and maintain scalable ETL/ELT pipelines processing millions of records.
• Map and reconcile complex legacy data structures across multiple business entities.
• Develop data validation and reconciliation frameworks to ensure data integrity and completeness.
• Optimize PostgreSQL performance through query tuning, indexing, partitioning, and batch/incremental loading.
• Troubleshoot data quality issues, schema mismatches, and pipeline failures.
• Document data mappings, transformation logic, data lineage, and migration runbooks.
• Collaborate with business and technical stakeholders on migration requirements and timelines.
• Establish monitoring, logging, and alerting for pipeline health and data quality.
• Mentor junior data engineers and contribute to engineering best practices.

• 7+ years of experience in Data Engineering.
• Strong to expert-level Python skills.
• Deep hands-on experience with PostgreSQL and/or Oracle.
• Proven experience designing and managing ETL/ELT pipelines at scale.
• Experience with large-scale data migration projects involving millions of records.
• Strong experience with data mapping, transformation, validation, and reconciliation.
• Knowledge of data modeling, normalization, indexing, and partitioning.
• Experience with Git and CI/CD practices.
• Strong troubleshooting and problem-solving skills.

• Experience with Airflow, Dagster, or Prefect.
• Experience with AWS, GCP, or Azure.
• Exposure to NoSQL and cross-database migrations.
• Experience working in regulated or high-stakes data environments.
• Experience with Docker and Infrastructure-as-Code.
• Familiarity with Great Expectations, dbt tests, or similar data quality frameworks.

• Fully migrated and validated datasets in PostgreSQL with zero critical data loss or corruption.
• Documented, repeatable, and optimized ETL/ELT pipelines.
• Clear data lineage and audit trails for migration and validation activities.