Data Engineer

Pay Range: $55 - $60 Per Hour

Job Description:

• Review and translate requirements to end-end technical design with a goal of building data products
• Develop pipelines in Databricks using PySpark and DBT to onboard various sources
• Working knowledge in Kafka to stream the data into databricks platform
• Identify Common approaches and best practices to data integration such as metadata drive ingestion, data validations, complex data type handling
• Drive standards for reusability, while developing pipelines
• Create or update documentation associated to key decisions on requirements, design and development
• Liaise with architects, data owners and users for reviews on day-day basis to ensure the quality of deliverables are high.

Skill Proficiency Requirements:

• 8+ years of overall IT experience with 6 years as a Data Engineer
• Experience in providing data engineering solutions
• Working knowledge of below technology toolsets:

o Cloud (Azure/AWS)

o Databricks, Data Mesh, Data as a Product

o Python

o Data streaming (Kafka, EventHub, Kinesis, SNS)

o Cloud-based API management - good to have

o Immuta - good to have

o Multi-cloud and on-prem application integration - good to have

Disclaimer: Diverse Lynx LLC is an Equal Opportunity Employer. All applicants and employees are evaluated without discrimination, based solely on their qualifications, ability, competence and performance. This email and its attachments may contain confidential or proprietary information and is intended only for the recipient(s). If you received this message in error, please disregard it and notify the sender. If you no longer wish to receive our communications, you may unsubscribe at any time.

Security Notice: Our official website is www.diverselynx.com We do not operate or authorize any other websites representing Diverse Lynx LLC.