Data Engineer Lead

Integrity. Care. Trust. Compassion. Expertise. Do these words resonate with you? These values of Builders culture create success in all we do. We strive to provide deeply supportive partnerships to our customers, agents, and each other. Builders is proud to be named among the Great Places to Work. Our award-winning culture has earned top marks in Company Direction, Employee Appreciation, Work-Life Balance, Leadership, and Compensation and Benefits. Our strong culture keeps us Built Strong in a forever-changing world, and our AM Best A Rating is evidence of our financial strength. Position Summary The Data Engineering Lead serves as the technical owner of the companys Azure Databricks lakehouse platform supporting its property and casualty insurance business. This role drives the design, development, and evolution of the platform, leveraging Databricks technologies including Delta Lake, Unity Catalog, Lakeflow pipelines, and Databricks SQL. The Data Engineering Lead establishes and promotes engineering standards, best practices, and architectural principles to ensure a scalable, secure, reliable, and high-performing data platform. A key responsibility is to lead the phased migration and modernization of data from Guidewire and legacy policy, claims, and billing systems into the Azure Databricks lakehouse. The role champions effective data governance, security, and lineage through Unity Catalog, partnering closely with the Data Governance Lead to translate governance policies into practical, consistent engineering practices. In addition, the Data Engineering Lead provides technical direction, mentorship, and leadership to a team of engineers and contractors, fostering collaboration, engineering excellence, and the successful delivery of strategic data initiatives. Responsibilities Design the Databricks lakehouse architecture (bronze/silver/gold medallion on Delta Lake) for property and casualty policy, rating, claims, billing, reinsurance and customer data.Plan and lead the phased data migration from Guidewire (PolicyCenter, ClaimCenter, BillingCenter) and legacy systems into the lakehouse on Azure Databricks, including historical loads, incremental/CDC (change data capture) loads, reconciliation and decommissioning support.Model property and casualty insurance data: accounts, policies, insured risks and exposures, coverages, endorsements, premiums, claims, reserves, payments and recoveries.Deliver data for Actuarial and Finance: written/earned/unearned premium, loss triangles, IBNR (incurred but not reported) inputs, loss ratios, Schedule P and NAIC (National Association of Insurance Commissioners) statutory reporting.Integrate third-party data sources such as geospatial and weather data.Define the Unity Catalog design: metastore, catalogs, schemas, environments (development, test and production), naming conventions, permissions, row/column-level security and lineage.Manage pipeline orchestration and operations in Databricks: design job scheduling and dependencies with Databricks Workflows (Lakeflow Jobs), set up retries, alerting and SLA (Service Level Agreement) monitoring, and establish failure handling and runbooks for production pipelines.Manage performance and cost: Spark tuning, partitioning and liquid clustering, Photon, cluster and SQL warehouse sizing.Shape the data infrastructure with IT (Information Technology) and the Platform Engineer: storage, networking, security, identity, IaC (Infrastructure as Code) and CI/CD (Continuous Integration/Continuous Deployment).Set engineering standards for pipelines, testing, code review and documentation; lead, mentor and review the work of Data Engineers and junior contractors.Partner with the data governance lead to implement access controls, PII (personally identifiable information) masking, lineage and data quality rules.Perform other duties as assigned. Qualifications Associate degree or an equivalent combination of education and experienceTen or more years of experience in data engineering, including six or more years of hands-on experience with Databricks (PySpark, Spark SQL, Delta Lake); experience leading or mentoring engineering teamsCertification as a Databricks Data Engineer Professional; certification as an Azure data or solutions architect is preferredKnowledge of Unity Catalog: governance model, permissions, row/column-level security, lineage and system tablesExperience leading large data migrations from core insurance and legacy systems into a lakehouse or data warehouse; experience with property and casualty insurance data; understanding of the policy and claims lifecycle and core property and casualty KPIs (key performance indicators)Knowledge of data modeling: dimensional modeling, data vault, SCD (slowly changing dimension) patterns and insurance data modelsSkill in technical aspects : Spark internals and performance tuning, streaming (structured streaming, auto loader), CDC patterns, MERGE, Lakeflow/Delta Live TablesSkill at the expert level in SQL (structured query language) and Python; software engineering practice ability (testing, code review, modular design)Knowledge of data infrastructure: ADLS (Azure data lake storage) Gen2, Azure Data Factory, Event Hubs, Key Vault, Entra ID, VNets (Virtual Networks) and Private LinkExperience scheduling and monitoring production pipelines with Databricks Workflows and/or Azure Data Factory, including job dependencies, parameterization, alerting and monitoring with Databricks system tablesExperience with Git-based version control (GitHub or Bitbucket), including branching strategies, pull request reviews, and Databricks Git foldersInfrastructure-as-code and CI/CD for data (Terraform, Databricks Asset Bundles, Azure DevOps, GitHub Actions or Bitbucket Pipelines)Experience with Guidewire data (Guidewire DataHub/InfoCenter, Cloud Data Access); supporting actuarial reserving and pricing for property and casualty lines of business; API (application programming interface) and integration tools such as MuleSoft is preferredKnowledge of insurance data standards such as ACORD (Association for Cooperative Operations Research and Development) is preferred Lets talk benefits! Competitive SalaryBonus StructureProfit SharingMedical, Dental, Vision InsuranceEmployer Paid Short Term DisabilityEmployer Paid Long Term DisabilityEmployer Paid Life InsuranceVoluntary Life Insurance401K with Company MatchPTO About Builders Builders is a mid-sized mutual with remarkable strengths. Rated A by AM Best, Builders has forged rock-solid financial strength and a reputation for reliability and fairness in fulfilling our promises to customers. Kind, collaborative, and customer-centric, our experienced and passionate teams foster a rewarding atmosphere of excellence, trust, and mutual respect, meriting the Culture Excellence honors from Top Workplaces. Flexible and highly personal, our experts leverage deeply supportive partnerships with knowledgeable independent agencies to drive better services and protection for policyholders. Our financial excellence, amazing people, and powerful partnerships build outstanding outcomes and peace of mind for our agents and their clients. This is what we mean by Insurance Built Strong. Builders Insurance Group is an Equal Opportunity Employer.We welcome applicants from all walks of life and don't discriminate based on any protected status. Join us in creating a diverse and inclusive workplace!