Azure Data Engineer

Skill Set

Databricks, Python, PySpark, Scala, SQL, Azure, Data warehousing

Job Description

Databricks Engineer: 9+ years of total experience with 5+ years relevant experience in the mandatory skills.
Mandatory Skills: Databricks, Hadoop, Python, PySpark, Scala, SQL, Azure/AWS cloud services

Required Skills & Experience:

• Very strong hands-on experience in Databricks with Azure cloud service in Data engineering/Data processing development projects implementing on Data warehouse.
• Hands-on experience in Azure/AWS Cloud-based development and integration
• Proficiency in programming languages Scala and Spark Data frame for data processing and application development
• Strong knowledge and experience in Facts, Dimensions, data Partitions, and handling large volume of data.
• Experience in handling large data in Databricks and performance optimization as well
• Practical experience with Data Engineering, Data Ingestion/Orchestration and the accompanying DevOps with CI-CD tools
• Strong knowledge of Spark, Databricks SQL - Data engineering pipeline implementing in Data warehouse
• Experience in offshore/onshore model and Ability and agile methodology.
• Gathering requirements, understand the business need and regular discussion with tech on design, development activities.
• Should have good experience working with client architect/design team to understand the architecture, requirements and work on the development.
• Experience working in a Financial Industry.
• Certification on Databricks and Azure will be added advantage

Work Location

Boston (MA)

BGC Post or Before Onboarding

BGC Before onboarding. There is a separate BGC from client side.
Client BGC and onboarding must need to be complete to start in the project.

Must Have Skills

Databricks, Scala, PySpark, Python, Data warehousing
Good Communication Skills

Work Model

Hybrid (4 days to work from Client office at Boston)