Data Engineer 3

5-10 Years Focused on manipulating data in a software engineering capacity. Some of that data might live in relational systems, but it is increasingly moving towards NoSQL systems and data lakes. Normalize databases and ascertain the structure of the data meets the requirements of the applications that are accessing the information. Construct datasets that are easy to analyze and support company requirements. Combine raw information from different sources to create consistent and machine-readable formats. This IT role requires a significant set of technical skills, including a deep knowledge of SQL, data modeling, and tools like Spark/Hive/Airflow. Must have skills and experience required for creating Databricks pipelines for multiple types of sources. * Working experience with batch and real-time data processing frameworks. * Working experience with data modelling, data access, schemas, and data storage techniques. * Working experience with data quality tools. * Experience in creating functional and technical designs for data engineering and analytics solutions. * Experience implementing data models of different schemas and working with diverse data source types. * Hands-on experience developing solutions with big data technologies such as Hadoop, HIVE and Spark. * Hands-on experience developing and supporting statistical models, R, and/or Python based AI/ML solutions. * 5+ years hands on experience designing, developing, testing, deploying, and supporting data engineering and analytics solutions using on premises tools such as MSBI (SSIS/SSAS), Informatica, Oracle Golden Gate, SQL, Oracle, and SQL Server. * 3+ years hands on experience designing, developing, testing, deploying, and supporting data engineering and analytics solutions using Microsoft cloud based tools such as Azure Data Lake, Azure Data Factory, Azure Databricks, Python, Azure Synapse, Azure Key Vault, and Power BI. * Experience with Containerization methodologies Docker, OpenShift, etc. * Experience with Agile as well as DevOps, CI/CD methodologies. * Hands-on experience designing and developing solutions involving data sourcing, enrichment and delivery using APIs & Web Services.