Big Data Architect Contractor

Position Summary

Contractor

Remote

Work you'll do

We are seeking a Big Data Architect Contractor to design, build, and optimize large-scale data solutions on Azure and Databricks. The role will focus on developing robust ETL/ELT pipelines, enabling high-performance data processing, and supporting end-to-end architecture across development, UAT, and production environments. The ideal candidate will work closely with client stakeholders and engineering teams to deliver scalable, secure, and cost-efficient data platforms.

Roles and Responsibilities

• Design, develop, and optimize ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks
• Build and tune PySpark and Spark SQL notebooks for large-scale data transformation
• Support end-to-end data solutions across dev, UAT, and prod environments using Unity Catalog
• Participate in technical design discussions with client architects and cross-functional counterparts
• Collaborate with teams on data contracts and schema agreements
• Support the design and optimization of high-volume data pipelines
• Define and enforce engineering standards, including naming conventions, partitioning strategies, cluster configurations, and Spark tuning
• Drive performance optimization through AQE tuning, liquid clustering, broadcast joins, and shuffle partition management
• Support Databricks cluster policies, autoscaling configurations, and cost optimization strategies
• Perform root cause analysis on production incidents and implement permanent fixes
• Support junior and mid-level engineers through code reviews and pair programming
• Evaluate emerging technologies and recommend adoption where relevant, such as DABs, DLT, Auto Loader, Serverless Compute, and Event Hubs

Key Skills

• Databricks - Advanced (6-9 years of experience)
• Strong hands-on experience with Azure Data Factory
• Advanced proficiency in Python, PySpark, and Spark SQL
• Strong background in big data architecture and scalable data platform design
• Experience with Unity Catalog and multi-environment delivery
• Knowledge of Spark performance tuning and optimization techniques
• Experience with cluster configuration, autoscaling, and cost management
• Strong analytical and troubleshooting skills for production support
• Ability to collaborate with technical teams and client stakeholders
• Experience supporting engineering standards and best practices

The expected pay range for this contract assignment is $65 - $70 per hour will vary based on skills, experience, and location and will be determined by the third-party whose employees provide services to Deloitte.

Candidates interested in applying for this opportunity must be geographically based in the United States and must be legally authorized to work in the United States without the need for employer sponsorship.

We do not accept agency resumes and are not responsible for any fees related to unsolicited resumes.

Deloitte is not the employer for this role.

#LI-AR2

#Remote

Expected Work Schedule

Approximate hours per week

About Deloitte

Our inclusive culture empowers our people to be who they are, contribute their unique perspectives, and make a difference individually and collectively. It makes Deloitte one of the most rewarding places to work.