Senior AWS Big Data Platform Administrator (Hadoop/Kafka)

Our client, a IT Services and Consulting company, is looking for a Senior AWS Big Data Platform Administrator (Hadoop/Kafka) for their Saint Louis, MO location.

Responsibilities:

Cloud & Infrastructure (AWS)

• Design, deploy, and manage scalable, secure, and highly available infrastructure on AWS.
• Administer core AWS services: EC2, S3, EMR, EKS, IAM, VPC, CloudWatch, Lambda.
• Implement automated infrastructure provisioning using Infrastructure as Code (Terraform/CloudFormation).
• Optimize cost, performance, and reliability of cloud environments.
• Develop disaster recovery and backup strategies.

Big Data Platform Administration (Hadoop)

• Install, configure, tune, and maintain Hadoop ecosystem components (HDFS, YARN, Hive, HBase, Spark).
• Monitor cluster health, performance, and capacity planning.
• Troubleshoot cluster issues and ensure high availability using HA configurations (NameNode HA, etc.).
• Manage data ingestion and processing workflows.

Streaming Platform Administration (Kafka)

• Install, configure, and manage Apache Kafka clusters.
• Ensure high throughput, low latency, and fault-tolerant streaming systems.
• Manage topics, partitions, brokers, and retention policies.
• Perform Kafka cluster scaling, upgrades, and performance tuning.
• Implement monitoring using tools like Prometheus, Grafana, or Confluent Control Center.

System Operations & Monitoring

• Implement monitoring, logging, and alerting for all systems.
• Conduct root cause analysis and performance tuning.
• Ensure system security, access controls, and compliance standards.
• Maintain detailed documentation and operational runbooks.

Automation & DevOps

• Build CI/CD pipelines for infrastructure and platform deployments.
• Automate routine administrative tasks using scripting (Python, Shell, etc.).
• Collaborate with DevOps and engineering teams for platform improvements.

Requirements:

• Minimum 10+ years of experience in system administration, with at least 5+ years in big data and cloud environments.
• Proven experience managing large-scale Hadoop clusters and Kafka environments.
• Strong hands-on experience with AWS cloud platform.
• Agentic AI experience
• AWS services: AWS Administration, EC2, S3, EMR, IAM, VPC, CloudWatch, Lambda
• Big Data : Administration, Hadoop, HDFS, Hive, Spark,
• Kafka Administration
• Years of Experience: 14.00 Years of Experience

Skills:

• Category Name Required Importance Experience
• Data Management Hadoop Administration No 1

Why Should You Apply?

• Health Benefits
• Referral Program
• Excellent growth and advancement opportunities