Microsoft Fabric Data Engineer

Data Management and Storage:

• Design and implement data storage systems using Azure services like Azure SQL Database, Azure Data Lake Storage, and Azure Synapse.
• Ensure scalability, performance, and cost-effectiveness.

Data Integration and ETL (Extract, Transform, Load):

• Develop and implement data integration processes using Azure Data Factory.
• Extract data from various sources, transform it, and load it into data warehouses or data lakes.

Big Data and Analytics:

• Utilize big data technologies such as Apache Spark.
• Create data processing workflows and pipelines to support data analytics and machine learning applications.
• Build and maintain new and existing applications in preparation for a large-scale architectural migration within an Agile function.
• Monitor and optimize data pipelines and database performance to ensure data processing efficiency.
• Build interfaces for supporting evolving and new applications and accommodating new data sources and types of data.
• Document data engineering processes, data models, and pipelines to ensure transparency and maintainability.
• Bachelor's degree Computer Science or a related field.
• 5+ of experience in building data and analytics platform focused on Azure data and analytics solutions.
• Expertise in Azure services such as Azure SQL Database, Azure Data Factory, Azure Synapse Analytics, and Azure Data Lake Proficiency in:
• Software development and scripting languages
• ETL tools (e.g., SSIS, Azure Data Factory, Power BI Dataflow).
• Database management (MSSQL, Azure SQL Database, Azure Data Lake Storage, and Azure Synapse).
• Knowledge of data modeling and data warehousing concepts.
• Excellent problem-solving and troubleshooting abilities.
• Attention to detail and commitment to data accuracy.
• Experience with cloud-based data migration.
• Strong analytical and problem-solving skills, with ability to conduct root cause analysis on system, process or production problems and ability to provide viable solutions.
• Experience working in an Agile environment with Scrum Master/Product owner and ability to deliver.
• 3+ years of programming experience in Python/Pyspark.
• 7+ years of experience in Microsoft Azure.
• Knowledge of Jira, Confluence, SAFe development methodology & DevOps.