Data Engineer - L3 Support Engineer (Lead)

Requirement ID:93172-1
Title: Data Engineer - L3 Support Engineer (Lead)
Pay range: $55.00 - $60/hr
Location: Irvine, CA
Duration: 6 Months

Job Description:
We are seeking an experienced Lead Data Engineer L3 Support with strong expertise in Databricks, Spark, PySpark, SQL, and Data Pipeline Engineering to lead production support and data engineering initiatives. The ideal candidate will possess deep technical knowledge of large-scale data platforms, hands-on experience with performance tuning and optimization, and a proven track record of managing L3 support operations in enterprise environments.

The role requires strong leadership skills to oversee support activities, drive incident resolution, perform root cause analysis, and ensure the stability, reliability, and performance of critical data pipelines. Experience in the Asset Management, Investment Management, or Financial Services domain is highly preferred.

Key Responsibilities

Data Engineering & Platform Support

• Lead L3 support activities for enterprise data platforms and data engineering solutions.
• Design, develop, enhance, and maintain scalable data pipelines using Databricks, Spark, and PySpark.
• Monitor production environments and proactively identify performance bottlenecks.
• Perform advanced troubleshooting and root cause analysis for complex production issues.
• Ensure data quality, integrity, reliability, and availability across the data ecosystem.
• Support batch and real-time data processing frameworks.

Performance Optimization

• Optimize Spark jobs, SQL queries, and Databricks workloads for maximum efficiency.
• Improve pipeline performance, scalability, and resource utilization.
• Analyze cluster performance and recommend tuning strategies.
• Implement best practices for code optimization and workload management.

Incident & Problem Management

• Lead critical incident resolution efforts and provide timely communication to stakeholders.
• Manage production support processes including incident management, problem management, and change management.
• Conduct root cause analysis (RCA) and implement preventive measures.
• Maintain operational runbooks and support documentation.

Leadership & Delivery Oversight

• Act as a technical lead for the data engineering support team.
• Mentor and guide junior engineers and support analysts.
• Coordinate with cross-functional teams including Data Engineering, Architecture, DevOps, QA, and Business stakeholders.
• Ensure adherence to SLAs, operational standards, and governance policies.
• Drive continuous improvement initiatives across support and engineering functions.

Domain Collaboration

• Work closely with Asset Management and business teams to understand data requirements and operational challenges.
• Support business-critical reporting, analytics, and investment data workflows.

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field.
• 8+ years of experience in Data Engineering, Production Support, or Data Platform Operations.
• Strong hands-on experience with:
  • Databricks
  • Apache Spark
  • PySpark
  • SQL
  • Data Pipeline Development
  • Performance Tuning and Optimization
• Experience supporting large-scale production data environments.
• Strong expertise in troubleshooting and resolving complex data platform issues.
• Experience with incident management, problem management, and root cause analysis.
• Excellent understanding of distributed data processing frameworks.
• Strong communication, stakeholder management, and leadership skills.

Preferred Qualifications

• Experience in Asset Management, Investment Management, Wealth Management, or Financial Services domains.
• Experience with:
  • Delta Lake
  • Azure Databricks
  • Apache Airflow
  • Azure Data Factory (ADF)
  • Python
  • CI/CD pipelines
  • DevOps practices
  • Cloud platforms (Azure, AWS, or GCP)
• Familiarity with monitoring and observability tools.
• Experience leading support teams or serving as a Tower Lead.

• Company Benefits & Culture
• Competitive salary and benefits package
• Supportive and collaborative work environment

• For immediate consideration please click APPLY to begin the screening process with Alex.

• Opportunities for professional growth and development