Lead Sr Software Engineer(Cloudera Hadoop stack)

Senior Software Engineer – Hadoop & AWS Big Data

Location:Blue Bell, PA
Employment Type: Full-Time

Position Overview
We are seeking an experienced Senior Software Engineer with strong expertise in Hadoop, AWS Big Data, and cloud migration to lead the design, development, and modernization of enterprise-scale data platforms. The ideal candidate will have extensive experience working with on-premises Hadoop environments and migrating large-scale data processing workloads to AWS cloud services.

This role requires close collaboration with software engineers, data engineers, DevOps teams, QA, Product Owners, and other stakeholders to build scalable, secure, and high-performance data solutions while mentoring team members and promoting engineering best practices.

Key Responsibilities

• Design, develop, and maintain high-quality, scalable software solutions and reusable components.

• Analyze business and technical requirements and translate them into detailed technical designs.
• Lead the modernization and migration of Hadoop-based platforms to AWS.
• Assess existing Hadoop architecture, workflows, and data pipelines.
• Design target cloud architectures using AWS big data services.
• Develop migration strategies for transferring data, applications, and processing workloads to AWS.
• Implement data ingestion, processing, storage, and analytics solutions.
• Conduct unit and integration testing to ensure solution quality.
• Troubleshoot software issues and provide production support.
• Create technical documentation, architecture diagrams, and migration runbooks.
• Ensure solutions meet performance, scalability, security, and compliance requirements.
• Implement automation using Infrastructure as Code (IaC).
• Mentor junior engineers and promote software engineering best practices.
• Collaborate with cross-functional teams throughout the software development lifecycle.

Required Qualifications

• Bachelor's degree in Computer Science, Information Technology, Information Systems, Engineering, or a related field.
• 10+ years of software engineering experience in enterprise applications and big data platforms.
• Strong experience with Hadoop ecosystem technologies including:
  • HDFS
  • Hive
  • Spark
  • Oozie
  • Python
• Extensive experience with AWS Big Data services, including:
  • Amazon EMR
  • AWS Glue
  • AWS Lambda
  • Apache Airflow
• Experience designing and implementing enterprise cloud migration strategies.
• Strong scripting experience using Bash/Shell.
• Experience with Infrastructure as Code (Terraform preferred).
• Strong knowledge of data security, governance, and cloud compliance.
• Experience with monitoring and logging tools.
• Familiarity with XML and JSON data formats.
• Strong problem-solving and analytical skills.
• Excellent communication and collaboration skills.

Preferred Qualifications

• Proven experience migrating large-scale Hadoop workloads to AWS.
• Experience with platform engineering and DevOps tools including:
  • Bitbucket
  • Gradle
  • Octopus Deploy
  • Artifactory
• AWS Certifications such as:
  • AWS Certified Solutions Architect
  • AWS Certified Big Data – Specialty
• Experience working with healthcare data, including:
  • Claims data
  • Provider data
  • Member data
• Experience building secure, scalable, and highly available cloud-native data platforms