We are looking for a highly skilled Tech Lead - Data Engineering to guide our engineering team in building and scaling a robust, modern data platform. In this role, you will be the bridge between architectural blueprints and engineering execution. You will remain deeply hands-on with our core stack-Databricks, dbt, and Apache Airflow-while mentoring engineers, conducting rigorous code reviews, and ensuring the delivery of high-quality data products.
Key Responsibilities
Technical Leadership & Delivery
Lead a team of data engineers to execute sprint goals, managing code quality and delivery timelines.
Act as the subject matter expert (SME) for the team on Databricks, dbt, and Airflow best practices.
Enforce engineering standards, including code modularity, documentation, and version control.
Conduct comprehensive code reviews to ensure scalability, security, and performance.
️ Hands-on Engineering & Optimization
Build and maintain production-grade data pipelines using PySpark, Delta Live Tables (DLT), and Spark SQL on Databricks.
Develop complex dbt models, custom macros, and tests to transform raw data into analytics-ready layers.
Author and schedule sophisticated Apache Airflow DAGs, utilizing dynamic task mapping and custom operators.
Optimize pipeline performance by troubleshooting bottlenecked Spark jobs, Z-Ordering Delta tables, and tuning Airflow schedulers.
DataOps & Governance
Implement and manage CI/CD pipelines for data deployments using toolsets like GitHub Actions or Azure DevOps.
Enforce data governance policies, access controls, and lineage tracking via Databricks Unity Catalog.
Integrate automated data quality testing and alerting mechanisms across dbt and Airflow workflows.
Mentorship & Collaboration
Mentor and upskill junior and mid-level data engineers through pair programming and workshops.
Translate architectural designs into actionable, granular engineering tasks and JIRA tickets.
Collaborate closely with data architects, product managers, and downstream data analysts.
Required Qualifications:
10+ years of professional experience in data engineering and backend software development.
2+ years of experience in a technical lead, team lead, or mentoring capacity.
Technical Proficiencies
Databricks: Strong hands-on experience with Delta Lake, Unity Catalog, and optimizing PySpark workloads.
dbt: Proficiency with dbt Core or Cloud, including advanced macros, packages, and custom testing.