Senior Data Engineer

Senior Data Engineer
Job Number: 26-00677

 
Find your next opportunity in the Financial Services Industry. ECLARO is looking for a Senior Data Engineer for our client in Remote, NY.
 
ECLARO’s client’s provides retirement savings, health & investment management and insurance services. If you’re up to the challenge, then take a chance at this rewarding opportunity
 
Position Overview:

• The Senior Data Engineer, leads the design and implementation of robust data solutions across multiple domains, driving technical excellence and scalability.
• This role mentor others, shape best practices, and influence data architecture.
• This role is expected to proactively identify opportunities to improve systems, drive reliability, and collaborate with product and business stakeholder to again data strategy and company goals.

• Design, build and scale robust, high-performing batch and real-time data pipelines.
• Drive architectural decisions for transformation logic, storage formats, and schema design.
• Lead complex data ingestion efforts and mentor peers on performance optimization and scalability.
• Lead the design and optimization of complex data models and storage architecture, balancing performance, scalability, and usability.
• Partner with stakeholders to translate business requirements into robust data structures.
• Contribute significantly to delivery planning and execution, mentor junior engineers on agile approaches, and ensure timely completion of tasks by managing dependencies and escalating delivery challenges.
• Design and standardize advanced data validation frameworks and testing strategies across platforms.
• Lead root cause analysis and data quality issues and mentor others on quality best practices.
• Partner with stakeholders to define SLAs and quality metrics.
• Leads efforts to automate, monitor, and scale deployment of production-grade data pipelines.
• Design resilient workflows with retry logic, failure handling, and resource optimization.
• Proactively address performance and reliability issues and contribute to runbooks and on-call documentation.
• Lead the creation and maintenance of detailed technical documentation for complex pipelines, data models, and system integrations.
• Establish and enforce documentation and development standards across projects.
• Mentor junior engineers on clear, consistent coding and documentation habits.
• Act as a key technical partner to product, analytics and data science teams.
• Lead design, discussions, communicate complex data trade-offs with clarity, and proactively surface risks and blockers.
• Support collaborative planning and mentor junior team members in effective communication and partnership.

• 5-9 years in data engineering, data modeling, and pipeline development
• Expert in SQL and Python for developing and debugging scalable data pipelines.
• Deep hands-on experience with Azure and Databricks, including Delta Live Tables and Unity Catalog
• Skilled with data integration/orchestration tools (SnapLogic, Azure Data Factory, Jenkins)
• Strong use of infrastructure-as-code tools like Terraform to manage deployment pipelines.
• Design and optimize API integration in pipelines.
• Familiar with data quality observability tools such as Soda or similar.
• Proficient in version control and CI/CD workflows using GitHub.
• Advanced understanding of dimensional modeling and data warehousing concepts.
• Comfortable leading efforts in agile environments and strong ownership and collaboration.

• 401k Retirement Savings Plan administered by Merrill Lynch
• Commuter Check Pretax Commuter Benefits
• Eligibility to purchase Medical, Dental & Vision Insurance through ECLARO