Senior Data Engineer / ML Platform Engineer (Azure Databricks & GenAI)

• Design and build production data pipelines on Databricks (PySpark/SQL, Delta Lake, Unity Catalog, Workflows/Jobs) with idempotency, testing, monitoring, and clear lineage.
• Model bronze / silver / gold layers and data contracts (schemas, SLAs, quality checks) for analytics and downstream applications.
• Develop ML-oriented data products: feature stores or curated tables, embedding generation, chunking/indexing for RAG, batch classification/enrichment, and evaluation of retrieval/data quality.
• I ntegrate with Azure services (ADLS Gen2, Key Vault, managed identities, private connectivity) following enterprise security and compliance patterns.
• Partner with application and ML teams to expose stable APIs, tables, and volumes that p ower agentic workflows (tool-backed assistants, multi-step orchestration) without owning full UI or real-time serving unless scoped.
• D ocument data dictionaries, runbooks, and handoff materials; participate in code review and agile ceremonies with full-time staff.

• 5 + years in data engineering, including 2+ years production Databricks on Azure.
• S trong Python (PySpark, pandas); SQL; experience operationalizing jobs at scale.
• Hands-on Delta Lake and Unity Catalog (schemas, grants, environments, promotion patterns).
• E xperience building pipelines that feed ML/GenAI use cases: embeddings, vector or hybrid search inputs, labeled datasets, batch inference outputs.
• Git, CI-friendly repo layout (notebooks + packages), unit/integration testing for data pipelines.
• Strong communication; effective in staff augmentation models (embedded with internal l eads, shared standards).

• GenAI / agentic systems: RAG design, retrieval evaluation, metadata for grounding, guardrails-aware data prep.
• Multi-source integration: documents, PLM/BOM-style structured data, files in object storage, APIs, spreadsheets normalized into a catalog.
• Exposure to containerized or microservice backends (for collaboration with app teams, not primary ownership).

• MLOps on Databricks (Model Serving, Feature Serving, experiment tracking).
• Stream or near-real-time ingestion (Structured Streaming, event-driven triggers).
• Infrastructure-as-code literacy (Terraform/Bicep) often owned internally; ability to consume outputs is enough.
• Graph or highly relational domains (requirements traceability, product structure, hierarchies).
• Automotive or complex manufacturing data (helpful, not required).

• Platform expertise, Databricks/Azure standards, architecture review, and access to tooling and repos per assignment.
• Clear priorities from product/engineering owners; expectations to align with existing team patterns rather than invent a separate stack.

careers@tekfortune.com