Data Engineer

At Staffworxs, we don't just connect talent we power transformation. Headquartered in Frisco, TX, with teams in Bengaluru and Hyderabad, we combine global reach with deep expertise. Our Digital & Data Analytics practice drives growth and innovation for some of the world's top Client brands, who continue to retain us as their trusted partner. If you're ready to make an impact, you're in the right place.

JOB DETAILS:
Job Title: Data Engineer
Location: Remote
Duration: Contract
Pay Rate: Open, DOE
Client Industry: Logistics

Job Description:

We are seeking an experienced Data Engineer expert to lead and support the SIT and UAT for migration of legacy Ab Initio & Teradata workloads to Azure Databricks. The candidate will serve as the onshore technical lead with hands on testing, defect fixing and performance tuning for the migrated workloads. Work closely with stakeholders and offshore delivery teams while ensuring data accuracy and delivery quality throughout the migration.

Roles & Responsibilities

• Develop Databricks notebooks, jobs, and workflows to replicate and enhance Abinitio/Teradata pipelines and transformations.
• Implement Delta Lake tables and patterns (bronze/silver/gold, ACID, time travel, schema evolution) for migrated data.
• Construct and optimize Databricks notebooks, workflows and Spark jobs leveraging PySpark and Spark SQL for advanced data transformation and analytics.
• Participate in data validation and reconciliation between Abinitio/Teradata and Databricks outputs.
• Provide defect fixes and support during SIT/UAT and post go-live stabilization.
• Maintain detailed technical documentation and adhere to coding standards, testing methodologies, and release management processes to ensure consistency and maintainability.

Required Skills & Expertise:

• Strong proficiency with Databricks platform, PySpark, Spark SQL, and Delta Lake architectures.)
• Solid understanding of data warehousing concepts, cloud-based data architectures, and dimensional data modeling methodologies.
• Fluency in SQL and Python with practical experience in development and debugging.
• Experience in performance tuning, workload optimization, and implementing cost-effective design solutions in data engineering pipelines.
• Familiarity with DevOps practices, CI/CD pipelines, version control systems, and automated testing as they relate to data engineering workflows.
• Strong analytical skills with an ability to communicate technical concepts effectively to both technical and business audiences.
• 3 to 5 years of experience in Databricks Implementations
• Certifications in Databricks will be an added advantage

Staffworxs is an Equal Opportunity Employer. We celebrate diversity and are committed to creating an inclusive workplace for all employees, regardless of race, color, religion, gender, sexual orientation, national origin, age, disability, or veteran status.