Must have experience with similar platform engineering/management solutions:
• Building/optimizing Data LakeHouse with Open Table formats
• Kubernetes deployments/cluster administration
• Transitioning on-premise big data platforms to scalable cloud-based platforms like AWS
• Distributed Systems, Microservice architecture, and containers
• Cloud Streaming use cases in Big Data Ecosystems (e.g., EMR, EKS, Hadoop, Spark, Hudi, Kafka/Kinesis)
Must have hands-on experience with below tech stack:
• Git Hib and Git Hub Actions
• AWS – IAM, API Gateway, Lambda, Step Functions, Lake Formation, Glue (Catalog, ETL, Crawler), Athena, S3 (Strong foundational concepts like object
data store vs block data store, encryption/decryption, storage tiers etc)
• Python
• EKS & Kubernetes, Apache Hudi, Flink, PostgreSQL and SQL, RDS
• Java background
• Terraform Enterprise
Helpful tech stack experience would include:
• Helm, Kafka and Kafka Schema Registry, AWS Services: CloudTrail, SNS, SQS, CloudWatch, Aurora, EMR, Redshift, Iceberg, Vault, AWS Secrets manager