Data Engineer

Job Title: Data Engineer

Location: Remote

Duration: long term

Job ID: 4CI -7453

Required Qualifications:

• Design and develop SQL Server stored procedures for data archiving, purge eligibility, and execution (soft/hard deletes).
• Handle EBCDIC data processing, including conversion to ASCII/Unicode and validation of data integrity.
• Develop Python-based automation for orchestration, validation, reconciliation, and reporting.
• Implement data retention rules, legal holds, and safe purge strategies with rollback and restore capabilities.
• Optimize performance for large datasets using indexing, partitioning, and efficient transaction handling.
• Create and manage batch jobs in Control-M, including scheduling, dependencies, alerts, and rerun strategies.
• Monitor data pipelines, troubleshoot issues, and support production incidents with root cause analysis.
• Maintain audit trails, ensure compliance, and enforce data security standards.
• Collaborate with DBAs, data governance, and application teams to align with retention policies and business needs.
• Strong experience with Microsoft SQL Server (T-SQL, stored procedures, indexing, query tuning).
• Hands-on experience with Python for data processing and automation (ODBC connectivity, logging, error handling).
• Knowledge of EBCDIC data formats, conversions, and validation techniques.
• Experience with Control-M for batch scheduling and job management.
• Understanding of data archiving, retention policies, and purge frameworks.
• Experience working with large datasets and performance optimization techniques.
• Preferred: Experience with VLDB, table partitioning, and archival solutions.
• Preferred: Familiarity with mainframe data models, COBOL copybooks, and packed numeric formats.
• Preferred: Knowledge of CI/CD tools (DACPAC, automated testing) for database and Python deployments.
• Preferred: Experience with data governance and regulatory compliance (SOX, PCI, HIPAA, GDPR).
• Strong analytical, troubleshooting, and documentation skills.

sql server, Python, Performance Tuning, data integrity, partitioning, Stored Procedures, indexing, T-SQL, batch scheduling, Control-M, ETL automation, Transaction Management, job dependencies, EBCDIC (037/1047), Data Conversion (EBCDIC ↔ ASCII/Unicode), ODBC, VLDB, Data Lifecycle Management, Audit andamp; Compliance, Data Archiving andamp; Purging, Logging andamp; Monitoring