1 year contract, fully remote
Position Overview
We are seeking an experienced Data Science Consultant to lead and execute the migration of existing data science and machine learning workflows from Dataiku to Databricks . This role will focus on modernizing analytics pipelines, improving scalability, and aligning with cloud-native best practices.

The ideal candidate brings strong hands-on experience in both platforms, deep knowledge of ML lifecycle management, and the ability to refactor and optimize data science solutions for a distributed compute environment.

Key Responsibilities

Migration & Modernization

• Lead the end-to-end migration of data science projects from Dataiku to Databricks
• Translate Dataiku workflows (recipes, scenarios, models) into Databricks-native architectures (Spark, notebooks, jobs)
• Rebuild and optimize:
  • Data pipelines
  • Feature engineering workflows
  • Model training and scoring pipelines

Architecture & Engineering

• Design scalable solutions leveraging:
  • Apache Spark (PySpark/Scala)
  • Databricks Lakehouse architecture
  • GitHub repository
• Implement best practices for:
  • Modular code development
  • Data versioning and lineage
  • Reusable ML pipelines
• Optimize performance for distributed processing environments

Machine Learning & MLOps

• Migrate and operationalize models using:
  • MLflow(experiment tracking, model registry, deployment)
• Ensure reproducibility and governance of ML workflows
• Implement CI/CD pipelines for model training and deployment
• Enable automated retraining and monitoring

Data Integration

• Work with structured and unstructured data sources
• Rebuild ingestion pipelines using:
  • Delta Lake
  • Streaming (if applicable)
• Ensure data quality, reliability, and validation frameworks

Collaboration & Documentation

• Partner with data scientists, data engineers, and platform teams
• Provide technical guidance on best practices in Databricks
• Document migrated workflows, architecture decisions, and runbooks
• Train internal teams on new workflows and tools

Calibro Corp. is a professional staffing and services firm specializing in the technology areas of Project Management, Business Analysis, Infrastructure and Development. We offer great benefits to both salaried and hourly employees including medical, dental, vision, 401k with match. Calibro Corp is committed to providing the highest levels of professional services to job seekers and employers. Reach out to our staffing specialists to experience the difference!