Data Engineer

Job Title: Data Engineer

Location: Malvern, PA/ Charlotte, NC

Duration: Fulltime

Skills: AWS | Python

Salary: 120K-140K/Year

Roles and Responsibilities:

Must Have Technical/Functional Skills

Python, Glue ETL, Attunity, Pyspark, AWS (S3, IAM, Lambda, Cloud Formation)

Roles & Responsibilities

• Design, develop, and maintain scalable ETL/data integration pipelines using AWS Glue, PySpark, and Python.
• Develop and optimize data transformation workflows to process large datasets from multiple source systems.
• Utilize Attunity for real-time and batch data replication between on-premises and cloud environments.
• Build and manage data lakes and storage solutions using Amazon S3, ensuring data quality, security, and governance.
• Implement serverless data processing solutions using AWS Lambda and integrate with other AWS services.
• Configure and manage IAM roles, policies, and permissions to ensure secure access to AWS resources.
• Automate infrastructure provisioning and deployment using AWS CloudFormation.
• Monitor, troubleshoot, and optimize ETL jobs, data pipelines, and cloud resources for performance and reliability.
• Collaborate with business, analytics, and application teams to gather requirements and deliver data solutions.
• Support CI/CD processes, perform code reviews, and maintain technical documentation following best practices.

Generic Managerial Skills, If any

• Planning & Execution - Effectively prioritize tasks, estimate effort, and ensure timely delivery of project milestones while balancing multiple commitments.
• Stakeholder Management - Build strong relationships with business teams, clients, and leadership through proactive communication and expectation management.
• Communication - Convey technical and non-technical information clearly through meetings, presentations, documentation, and status updates.
• Collaboration - Foster teamwork across cross-functional groups, encourage knowledge sharing, and contribute to a positive team environment.
• Decision Making - Analyze options, assess risks, and make informed decisions that align with both technical and business objectives.
• Leadership & Ownership - Take accountability for deliverables, drive initiatives proactively, and lead by example to achieve team goals.
• Risk & Issue Management - Identify potential risks early, address blockers proactively, and implement mitigation strategies to ensure smooth project execution.
• Customer & Business Focus - Understand customer needs and business priorities, ensuring technical solutions deliver measurable value.
• Mentoring & Coaching - Support team growth through guidance, code reviews, knowledge sharing, and skill development initiatives.
• Project & Change Management - Adapt effectively to evolving requirements, manage priorities during change, and ensure successful delivery outcomes.

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.