Senior data engineer

Job Title

Senior Data Engineer

Job Summary

We are seeking an experienced Senior Data Engineer to design, develop, and maintain scalable data pipelines and data platforms that support business intelligence, analytics, machine learning, and operational reporting. The ideal candidate will have strong expertise in data architecture, ETL/ELT processes, cloud platforms, and big data technologies, along with the ability to mentor junior engineers and collaborate with cross-functional teams.

Key Responsibilities

• Design, build, and optimize scalable data pipelines for structured and unstructured data.
• Develop and maintain ETL/ELT workflows to ingest, transform, and load data from multiple sources.
• Design and implement data models, data warehouses, and data lakes.
• Ensure data quality, integrity, governance, and security across the organization.
• Optimize database performance, query execution, and data processing workloads.
• Collaborate with data scientists, analysts, product managers, and business stakeholders to understand data requirements.
• Implement monitoring, alerting, and troubleshooting mechanisms for data pipelines.
• Build and maintain cloud-based data platforms and infrastructure.
• Lead data architecture discussions and recommend best practices.
• Mentor junior data engineers and participate in code reviews.
• Support machine learning and AI initiatives by preparing reliable and scalable datasets.
• Document data processes, standards, and technical solutions.

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or a related field.
• 5+ years of experience in data engineering or related roles.
• Strong proficiency in SQL and database design principles.
• Experience with Python, Scala, Java, or similar programming languages.
• Hands-on experience with ETL/ELT tools and frameworks.
• Strong knowledge of data warehousing concepts and dimensional modeling.
• Experience with cloud platforms such as AWS, Azure, or GCP.
• Expertise in distributed data processing technologies such as Spark, Hadoop, or Databricks.
• Experience working with relational and NoSQL databases.
• Knowledge of CI/CD pipelines, DevOps practices, and version control systems.
• Strong analytical, problem-solving, and communication skills.

Preferred Qualifications

• Experience with data orchestration tools such as Apache Airflow.
• Experience with streaming technologies such as Apache Kafka or Kinesis.
• Familiarity with containerization and orchestration tools (Docker, Kubernetes).
• Experience supporting machine learning workflows and MLOps.
• Cloud certifications (AWS, Azure, or GCP) are a plus.

Technical Skills

Data Engineering

• SQL
• Python
• Apache Spark
• Hadoop
• Databricks
• ETL/ELT Development
• Data Warehousing

Cloud Platforms

• AWS (Redshift, Glue, EMR, S3)
• Azure (Data Factory, Synapse, Data Lake)
• GCP (BigQuery, Dataflow, Cloud Storage)

Databases

• PostgreSQL
• MySQL
• SQL Server
• MongoDB
• Cassandra

Tools & Technologies

• Apache Airflow
• Kafka
• Docker
• Kubernetes
• Git
• Jenkins
• Terraform

Preferred Experience

• Building enterprise-scale data platforms.
• Supporting real-time and batch data processing systems.
• Working in Agile/Scrum environments.
• Leading technical projects and mentoring engineering teams.

Key Competencies

• Data Architecture
• System Design
• Performance Optimization
• Leadership & Mentoring
• Stakeholder Management
• Problem Solving
• Communication & Collaboration

Experience: 5 10+ Years
Employment Type: Full-Time
Location: Remote/Hybrid/Onsite