Request ID: 111240-1
Title: Senior Data Engineer
Locations: Malvern, PA
Interview Mode: Onsite
Duration: 06+ Months with Possible Extension.
Pay Range: $60-$65/Hour on W2/C2C (All inclusive)
We are seeking an experienced AWS Data Engineer with expertise in ETL processes, AWS Lambda, Redshift, and proficiency in Python and PySpark. This role requires hands-on experience with cloud-based architecture, data engineering, and working with large-scale data processing systems. The ideal candidate will play a critical role in building and maintaining scalable, efficient, and secure ETL pipelines within the AWS ecosystem.
Key Responsibilities:
" Design, develop, and maintain robust ETL workflows to extract, transform, and load data from various sources to AWS Redshift and other cloud-based systems.
" Leverage AWS Lambda to build serverless computing solutions for real-time data processing and automation.
" Work with AWS data storage and processing services, including S3, Redshift, Athena, and Glue, to support data processing pipelines.
" Utilize Python and PySpark to create and optimize data transformation logic and analytical queries.
" Collaborate with cross-functional teams to understand data requirements and provide efficient data solutions.
" Monitor and optimize cloud-based infrastructure for performance, reliability, and cost efficiency.
" Troubleshoot and resolve data pipeline issues, ensuring minimal downtime and high data accuracy.
" Implement automation and monitoring tools to enhance reliability.
Required Skills and Qualifications:
" Proven experience working with AWS services, including Lambda, Redshift, S3, Athena, Glue, and CloudWatch.
" Strong knowledge and hands-on experience with Python and PySpark for data transformation and processing.
" Expertise in building and optimizing ETL pipelines in a cloud environment.
" In-depth understanding of relational databases, data warehousing, and performance tuning in Redshift.
" Strong problem-solving and troubleshooting skills.
" Familiarity with infrastructure as code tools like AWS CloudFormation.
" Experience with version control (e.g., Git) and CI/CD pipelines.
" Solid understanding of data security, data privacy, and compliance frameworks.
" Ability to work in an agile, fast-paced environment.
Ex-Client and local Candidates preferred
Skills: Digital: Python~ Digital : Amazon Web Service (AWS) Cloud Computing~ Digital: PySpark~ Core Java
Experience Required: 10 & Above