Job Description
Position Overview
We are seeking a Senior Data Engineer to design, build, and operate cloud-scale data platforms and pipelines that support analytics, machine learning, and data products across the organization. This role collaborates with Product, Architecture, and Data Science teams to deliver secure, reliable, and cost-efficient batch and streaming data services. The ideal candidate brings deep technical expertise in cloud data engineering, data platform architecture, and best practices for governance, reliability, and observability.
Key Responsibilities
We are seeking a Senior Data Engineer to design, build, and operate cloud-scale data platforms and pipelines that support analytics, machine learning, and data products across the organization. This role collaborates with Product, Architecture, and Data Science teams to deliver secure, reliable, and cost-efficient batch and streaming data services. The ideal candidate brings deep technical expertise in cloud data engineering, data platform architecture, and best practices for governance, reliability, and observability.
Key Responsibilities
- Lead the design, development, and operation of enterprise data platforms, including lakehouse, streaming, and centralized data hubs, while managing dependencies and integrations with enterprise systems.
- Provide technical leadership and guidance to engineering teams, including planning, scoping, risk management, and delivery accountability.
- Coach engineers on standards, observability, operational excellence, and best practices;conduct design and code reviews.
- Define and evolve cloud data architectures for ingestion, storage, compute, cataloging, and access across real-time and batch use cases.
- Build and maintain robust ELT/ETL and streaming pipelines using orchestration frameworks (Airflow, Dagster), event platforms (Kafka, Kinesis, Event Hub), and distributed processing frameworks (Spark, Databricks, Flink, Beam).
- Implement lakehouse/lake patterns with medallion layering, CDC, and schema evolution;ensure quality gates, testing, and lineage.
- Establish and monitor service-level objectives for data freshness, availability, and quality;implement observability and runbooks for operations.
- Optimize storage and compute costs and performance;manage capacity, autoscaling, and governance.
- Ensure privacy, security, and compliance (IAM, tokenization, row/column security, data retention);work closely with enterprise security teams.
- Translate product and stakeholder requirements into data contracts and SLAs;document architecture and operational procedures.
- Evaluate emerging data technologies and provide proof-of-concepts and reference implementations.
- Anticipate future platform needs, including multi-cloud, IoT/edge ingestion, and AI/ML workloads.
- Modernize existing data architectures using current best practices: lakehouse, CDC, medallion layering, data mesh principles, and streaming pipelines.
- Develop software engineering solutions in Python/SQL and one additional language (Scala/Java);familiarity with REST/gRPC and microservices patterns.
- Utilize cloud data services (Azure Data Lake/Databricks/Synapse;AWS S3/EMR/Glue/Redshift;GCP BigQuery/Dataflow) and distributed stores (Delta/Iceberg/Hudi, Kafka, Redis).
- Work with SQL/NoSQL databases (Postgres, DynamoDB, MongoDB, Cassandra) and CI/CD pipelines, containerization (Docker/Kubernetes), and infrastructure as code (Terraform).
- Ensure proper data governance and security across catalogs, lineage, RBAC/ABAC, encryption, and key management.
- Communicate effectively with stakeholders, translating business needs into scalable, reliable data solutions.
- Bachelor’s degree in Computer Science, Engineering, or related field;Master’s preferred.
- 5–10+ years of experience in data engineering or platform roles delivering enterprise-grade systems (SaaS or large-scale internal platforms).
- 2+ years leading or architecting cloud data platforms at scale.
- Proven experience with batch and streaming pipelines, lakehouse implementations, and production ML data flows.
- Experience operating and supporting data platforms with SLAs/SLOs, cost governance, and on-call responsibilities.
- Hands-on experience with cloud-native data services, orchestration frameworks, and distributed processing.
- Experience with real-time data systems and IoT ingestion.
- Background in large-scale SaaS or multi-cloud platforms.
- Strong collaboration skills with cross-functional teams, including product, architecture, and analytics.
Numbers & Facts
| Location | Houston, TX |
| Industry | Staffing/Employment Agencies |
| Company Size | 100 to 499 employees |
| Website | https://www.prg-usa.com/ |
About Company
Established in 2001, Peyton Resource Group is a solution-based staffing company that matches businesses with top talent for short-term, long-term or permanent needs.
People are a business’s most valuable asset. Peyton Resource Group is dedicated to helping companies find the best talent, matching professionals with jobs where they will thrive.
With locations in Dallas/Fort Worth, San Antonio and Austin, we are available to serve your staffing needs throughout Texas and across the country.
Skills
Level up your application
Professional resume templates
Browse dozens of recruiter approved resume templates, layouts and formats. Choose your favorite and make it your own in minutes.
Free resume templatesFree resume builder
Improve your existing resume or start from scratch and create a standout, ATS-friendly resume. Add job-specific content, download and apply.
Free resume builder