Design and implement end-to-end data architecture / data ecosystem using Azure-native services and Databricks
Build scalable, high-performance data platforms and robust data frameworks
Partner with stakeholders to translate business requirements into reliable data solutions
Implement data governance principles using Unity Catalog
Create and implement data ingestion and data integration patterns using Medallion Architecture
Apply best practices in data engineering
Develop PySpark notebooks
Create procedures for advanced data transformation
Handle performance tuning in Databricks and Azure SQL DBs
Architect end-to-end data solutions supporting:
Yield improvement
Process control
Predictive maintenance
Operational efficiency
Ensure strict governance, security, scalability, and compliance across the platform
Required Experience
10+ years of overall experience and 5+ years of architecture experience in data architecture / data engineering roles
Strong hands-on experience with major enterprise data platforms
Extensive experience with Azure-native services including:
Azure Data Lake Storage (ADLS)
Azure Data Factory (ADF)
Azure Synapse Analytics
Azure SQL
Strong hands-on experience with Azure Databricks
Required Databricks Expertise
Should have worked with the latest Databricks capabilities including:
Unity Catalog:
Catalog and schema design
Permissions and security
Managed and external tables
Lineage and governance
Delta Live Tables (DLT):
Reliable pipeline development
CDC implementation
Data transformations
Data quality controls
Scaling and performance tuning
Other modern Databricks capabilities and best practices
Data Ingestion & ELT Experience
Experience with data ingestion tools such as:
Fivetran
HVR or equivalent CDC / replication tools
Experience with ELT tools such as:
DBT
Familiarity with:
Real-time / streaming data
Batch processing
Sensor / OT / ET data flows
Requirements
Technical Skills
Strong expertise in data ingestion, transformation, and orchestration frameworks
Hands-on experience building insurance-specific data models
Proficiency in Python / PySpark and Spark SQL
Strong understanding of streaming vs batch processing
Experience with Spark job performance tuning
Ability to optimize:
Data storage formats
Partitioning strategies
Schema design
Experience working with:
Object stores
Data lakes
Cloud compute scaling
Databricks cluster management
Governance, Security & Operations
Deep understanding of:
Data governance
Security and compliance
RBAC and attribute-based access control
Encryption and audit logs
Data privacy requirements
Experience with operational excellence practices:
Monitoring and observability
Reliability and metrics
Backup, failover, and disaster recovery strategies
Preferred Qualifications
Prior experience working on Insurance P&C cloud-based data and analytics projects
Strong analytical and problem-solving skills
Experience defining and enforcing best practices for Databricks workloads and Spark-based pipelines
Solid understanding of distributed data processing concepts
Strong communication and stakeholder management skills
Ability to document architecture decisions, trade-offs, and technical recommendations
Benefits
Diversity Inclusion: Exavalu also promotes flexibility, adapting to the needs of employees, customers, and the business. It might be part-time work, working outside normal 9-5 business hours or working remotely. We also offer a welcome back program to help individuals return to the mainstream after a prolonged absence due to health or family reasons. At Exavalu, we are committed to building a diverse and inclusive workforce. We welcome applications for employment from all qualified candidates, regardless of race, colour, gender, national or ethnic origin, age, disability, religion, sexual orientation, gender identity or any other status protected by applicable law. We nurture a culture that embraces all individuals and promotes diverse perspectives, where you can make an impact and grow your career
Numbers & Facts
Location
Dallas, Texas (Remote)
Website
www.exavalu.com
Skills
Access Controlunmatched
Analysis Skillsunmatched
Best Practicesunmatched
Business Operationsunmatched
C Programming Languageunmatched
Centers for Disease Control and Prevention (CDC)unmatched
Cisco Unityunmatched
Cloud Computingunmatched
Communication Skillsunmatched
Data Analysisunmatched
Data Formatsunmatched
Data Lakeunmatched
Data Modelingunmatched
Data Processingunmatched
Data Qualityunmatched
Data Storageunmatched
DataArchitect Data Modeling Toolunmatched
Database Designunmatched
Ecosystemsunmatched
Failoverunmatched
Insuranceunmatched
Metricsunmatched
Microsoft Windows Azureunmatched
Multiplatform/Cross-Platformunmatched
Performance Tuning/Optimizationunmatched
Problem Solving Skillsunmatched
Procedure Developmentunmatched
Process Control Engineeringunmatched
Productivity Managementunmatched
Python Programming/Scripting Languageunmatched
Replication and Remote Mirroringunmatched
Requirements Managementunmatched
SQL (Structured Query Language)unmatched
SQL Databasesunmatched
Scalable System Developmentunmatched
🎯
Be found by employers
5,500+ employers search our resume database daily. Add yours to get found by recruiters looking for candidates like you.
Level up your application
Professional resume templates
Browse dozens of recruiter approved resume templates, layouts and formats. Choose your favorite and make it your own in minutes.