Iceberg DBA / Lakehouse Operations Engineer

Iceberg DBA / Lakehouse Operations Engineer

Role Iceberg DBA / Lakehouse Operations Engineer

Mode of work: Remote

6+ Months

Job Summary:

We are seeking a highly skilled Iceberg DBA / Lakehouse Operations Engineer to own the reliability,

performance, and operational integrity of the Iceberg data layer powering enterprise analytics and

business-critical applications.

This role operates in a large-scale, multi-engine Lakehouse environment, supporting workloads across

Spark, Hive, and Impala, and plays a key role in enterprise data modernization initiatives (Hive and

Teradata Iceberg).

The ideal candidate brings deep expertise in Iceberg table operations, metadata management, and query

performance optimization, ensuring consistent, high-performance data access across platforms in a cloud

based environment.

This role is critical to ensuring data accuracy and performance-any degradation directly impacts

downstream reporting, analytics, and business-critical decision-making

Iceberg Data Layer Ownership & Operations

Own day-to-day operations of Apache Iceberg tables supporting multiple enterprise

applications

Ensure data reliability, consistency, and availability across all Lakehouse workloads

Maintain operational integrity for datasets at multi-terabyte to petabyte scale

Advanced Table Management & Optimization

Execute advanced Iceberg table maintenance and optimization strategies:

o Compaction (minor/major) and small file mitigation

o Snapshot expiration and metadata compaction to control metadata growth

o Orphan file cleanup (vacuum) to maintain storage efficiency

Optimize data layout and performance through:

o File size tuning and distribution strategies

o Partition evolution and pruning optimization

o Clustering and ordering techniques (e.g., Z-ordering or similar patterns)

Data Modeling Standards & Lakehouse Design Alignment

Support and enforce data modeling best practices aligned with:

o Normalized data structures (3NF) for source-aligned datasets

o Medallion architecture (Bronze / Silver / Gold layers) for curated data flows

Ensure Iceberg table design aligns with:

Iceberg DBA / Lakehouse Operations Engineer

o Data ingestion patterns (raw vs curated layers)

o Downstream consumption and performance requirements

Assist in structuring datasets to balance:

o Data integrity and normalization

o Query performance and analytical efficiency

Work with data engineering teams to ensure consistent implementation of layered data

architecture across multiple applications

Multi-Engine Query Performance & Consistency

Ensure consistent and performant query behavior across:

o Spark (CDE)

o Hive / Impala (CDW)

Troubleshoot and resolve:

o Query performance bottlenecks

o Metadata inconsistencies across engines

o Inefficient execution plans and scan patterns

Hive & Teradata Modernization Support

Play a key role in enterprise data platform modernization (Hive and Teradata Iceberg)

Support:

o Schema alignment and data type mapping

o Data validation and reconciliation

Troubleshoot migration-related issues and ensure post-migration stability and

performance

Metadata & Data Lifecycle Management

Manage Iceberg metadata to ensure:

o Efficient scaling and performance

o Consistent table state across engines

Execute lifecycle operations:

o Data retention and archival policies

o Snapshot lifecycle management and cleanup

o Time-travel optimization and maintenance

Production Support, Incident Resolution & On-Call

Provide L2/L3 support for data-related production issues across Iceberg-based

Lakehouse workloads

Participate in on-call rotation to support critical data platforms and ensure timely

response to incidents

Iceberg DBA / Lakehouse Operations Engineer

Respond to and resolve P1/P2 production incidents within defined SLAs, minimizing

impact to downstream applications and reporting

Troubleshoot:

o Data inconsistencies and reporting discrepancies

o Query failures and performance degradation

Perform root cause analysis (RCA) and implement preventive measures to avoid

recurring issues

Collaborate with platform and application teams during incident triage and resolution

Security & Data Governance Support

Support fine-grained access control using:

o Ranger policies and RBAC

Own and ensure data validation, reconciliation, and accuracy between source and

Iceberg datasets

Ensure secure and compliant access to data across applications

Required Skills

Strong hands-on experience with Apache Iceberg and/or Hive-based data lakes

Understanding of data modeling concepts (normal forms) and modern Lakehouse

patterns (Medallion architecture)

Expertise in:

o Table-level optimization and performance tuning

o Large-scale data management (TB/PB scale)

Experience with:

o Spark SQL, Hive, Impala, NiFI, Trino

Strong understanding of:

o Partitioning strategies

o File formats (Parquet/ORC)

o Distributed query processing

Preferred Skills

Experience with:

o Hive-to-Iceberg or Teradata-to-Iceberg migration

o Cloudera CDP (CDE/CDW)

Familiarity with:

o Cloud platforms (AWS, Azure)

Scripting/automation (Python, Shell)