Principal Architect (Databricks, Azure, PySpark)

Principal Architect

As a Principal Architect, you will leverage your extensive experience to design and implement cutting-edge data solutions using Databricks and Azure technologies. You will collaborate with cross-functional teams to optimize data workflows and ensure seamless integration within the retail domain. Your expertise will drive innovation and efficiency, enhancing our company's impact on the retail industry.

Responsibilities

• Lead the design and implementation of scalable data architectures using Databricks Unity Catalog and Delta Live Pipelines to enhance data management capabilities.
• Oversee the integration of Azure Data Lake Store and Azure DevOps to streamline data storage and deployment processes.
• Provide technical guidance in Python and PySpark to develop robust data processing solutions that meet business requirements.
• Collaborate with cross-functional teams to ensure seamless data integration and workflow optimization using Databricks SQL and Workflows.
• Drive innovation by implementing advanced data analytics solutions that support retail domain operations, including collaboration, planning, forecasting, and replenishment.
• Ensure data solutions align with industry best practices and company standards to maintain data integrity and security.
• Develop and maintain comprehensive documentation for data architectures and workflows to facilitate knowledge sharing and continuity.
• Monitor and optimize data processing performance to ensure efficient resource utilization and timely delivery of insights.
• Engage with stakeholders to understand business needs and translate them into technical requirements for data solutions.
• Contribute to the continuous improvement of data processes by identifying opportunities for automation and efficiency gains.
• Support the development of data governance frameworks to ensure compliance with regulatory requirements and company policies.
• Foster a collaborative environment by working closely with team members to share knowledge and best practices.
• Stay updated with the latest advancements in data technologies and methodologies to drive continuous innovation.