Data Engineer – Azure Databricks (Corporate Vertical Developer)
We are seeking an experienced Data Engineer – Azure Databricks (Corporate Vertical Developer) to support multiple corporate business functions, including RESO, HCG, Compliance, Facilities, and Finance. This role will be responsible for designing and building scalable data solutions, enabling enterprise-wide analytics, and driving data integration initiatives across the organization.
The ideal candidate will have strong expertise in Azure cloud technologies, Databricks, data engineering, and enterprise data integration, with the ability to collaborate effectively across business and technical teams.
Location: NYC Area (Hybrid – 3 days onsite per week)
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Microsoft Azure and Azure Databricks.
- Build and optimize ETL/ELT processes for both structured and unstructured data sources.
- Partner with stakeholders across corporate functions to gather requirements and deliver data-driven solutions.
- Design and implement data models, architectures, and frameworks that support reporting, analytics, and business intelligence initiatives.
- Integrate enterprise platforms such as ServiceNow, Dataverse, Workday, and other business applications into centralized data environments.
- Ensure adherence to data quality, governance, security, and compliance standards.
- Monitor, troubleshoot, and optimize data workflows to ensure high availability and performance.
- Contribute to the ongoing enhancement of enterprise data platforms and recommend innovative technologies and best practices.
- Support data modernization initiatives and enterprise-wide analytics strategies.
Required Qualifications
- Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field (or equivalent experience).
- 5+ years of experience in Data Engineering or a related discipline.
- Strong hands-on experience with Microsoft Azure data services.
- Extensive experience with Azure Databricks, including Apache Spark and distributed data processing.
- Advanced SQL development and optimization skills with large-scale data environments.
- Experience building and orchestrating data pipelines using Azure Data Factory (ADF) or similar ETL/ELT tools.
- Strong experience with Azure SQL Database, including performance tuning and optimization.
- Proficiency in Python and/or Scala for data engineering and automation.
- Experience with C# and PowerShell for integration and automation tasks.
- Experience developing and deploying Azure Functions for data processing and system integrations.
- Experience integrating APIs, event-driven architectures, and enterprise applications.
- Strong understanding of scalable, secure, and cloud-native data architecture principles.
- Excellent communication and collaboration skills with the ability to work across multiple business domains.
Preferred Qualifications
- Experience with Azure DevOps and CI/CD pipeline implementation.
- Knowledge of Delta Lake and modern Lakehouse architectures.
- Familiarity with data governance, metadata management, and data catalog solutions.
- Experience working within Agile/Scrum development environments.
- Relevant Microsoft Azure and/or Databricks certifications.
Flexible work from home options available.
Who We Are
High Bridge is a bottom-up consulting firm with more than two decades of experience creating innovative solutions for the technology industry.
- • We connect thoughtfully curated talent with reputable employers in our field.
- • We partner with like-minded companies to bolster mutual success and understanding.
- • We deliver results through strategic planning and meticulous project management.
At High Bridge, we believe that success is a measure of results and process, product and person. We are proud to bring integrity and humanity to the work that we do each day. Whether we’re interviewing prospective associates, brainstorming with partners, or troubleshooting with vendors, we pledge our commitment to honesty, transparency, and good humor at every junction.
Careers
Are you passionate about technology? Do you believe in no finish lines? Are people and process the heel in your stride? High Bridge Consulting may be the next journey for you.
Here at High Bridge, our island is technology, but our village is human. We value a culture of authenticity, integrity, and creativity. We work smart, not hard, we look forward, not backward, and we love coming to work on Monday.
If you aren’t afraid of heights, join us on our bridge.