Senior Data Engineer (ADF, Kafka)

Role: Senior Data Engineer
Location: Sunnyvale, CA – Onsite
Duration: 6 Months
Primary Skills: ADF, Kafka
 

Role Overview

We are seeking an experienced Senior Data Engineer with strong expertise in data strategy, transformation and integration, data pipeline development, data modeling, analytics, and data governance. The ideal candidate will be responsible for building efficient data solutions, developing and optimizing data pipelines, integrating diverse data sources, and translating business requirements into scalable technical solutions.
 

Key Responsibilities

• Apply data strategy principles to solve routine and complex business problems.
• Extract data from identified databases and develop scalable data pipelines.
• Transform data into structures relevant to business and analytical requirements.
• Identify appropriate data sources and perform initial data quality validation.
• Analyze complex data elements, systems, data flows, dependencies, and relationships.
• Develop conceptual, logical, and physical data models.
• Design and maintain data warehouses and data marts.
• Define relational tables, primary and foreign keys, and stored procedures.
• Evaluate existing data models and databases for discrepancies and optimization opportunities.
• Design efficient data flows and analyze system integration challenges.
• Develop code using appropriate programming languages and technologies.
• Create test cases, proofs of concept, and validate technical solutions.
• Perform software testing and deploy solutions to production environments.
• Maintain code documentation, playbooks, and provide regular progress updates.
• Translate business problems into data, analytics, automation, or mathematical solutions.
• Provide recommendations to business stakeholders for complex business challenges.
• Develop business cases focused on ROI, cost savings, and process improvements.
• Translate business requirements into actionable projects, activities, and technical tasks.
• Act as a bridge between business stakeholders and technical teams.
• Establish, modify, and document data governance processes and recommendations.
• Implement data governance practices in collaboration with business stakeholders.
• Ensure adherence to company and regulatory data policies.
• Educate stakeholders on data governance practices, policies, and guidelines.
• Recommend improvements to data processes, governance standards, and ways of working.
   

Required / Essential Skills

• Strong experience in Azure Data Factory (ADF).
• Strong experience with Apache Kafka and data streaming/integration.
• Data strategy and data transformation/integration.
• Data pipeline development and optimization.
• Data source identification and data quality validation.
• Conceptual, logical, and physical data modeling.
• Data warehouse and data mart design.
• Relational database design and stored procedures.
• Data flow design and optimization.
• System integration analysis and solution design.
• Strong code development and testing experience.
• Test case creation and proof-of-concept development.
• Production deployment and troubleshooting.
• Code documentation and technical playbook maintenance.
• Analytics, big data analytics, and automation.
• Business requirements translation and business case development.
• Data governance and regulatory/company policy interpretation.
• Strong stakeholder management and communication skills.
   

Preferred Skills

• Experience designing enterprise-scale data pipelines.
• Experience with real-time and event-driven data architectures.
• Strong understanding of data integration patterns and streaming technologies.
• Experience with data governance frameworks and data quality initiatives.
• Experience identifying process improvements and recommending new ways of working.
• Strong analytical, problem-solving, and business acumen.
  
   

Thanks,