Strategic Project Lead, DataFactory

JOB TITLE: Strategic Project Lead, DataFactory
LOCATION: Santa Clara, CA/ REMOTE
DURATION: 12 months
RATE RANGE: $60-$100/hr

COMPANY:
Our client is a global leader in AI and high-performance computing, powering innovations in gaming, robotics, and data science.

Job Description:
We are seeking a seasoned, techno-functional leader to drive the development and execution of large-scale LLM training programs. This role is deeply technical and customer-facing, with significant emphasis on dataset creation, annotation and workflows, and high-quality data pipelines for foundational model training. The ideal candidate combines deep technical understanding of the LLM training lifecycle, strong operational rigor, and exceptional cross-functional leadership. This candidate should also be comfortable operating with little structure, handling sensitive information responsibly, and navigating complex organizational dynamics with humility and confidence.

Key Responsibilities:

• Lead the generation and delivery of high-quality, scalable LLM training datasets with a focus on SFT, RLHF, rubric-based evaluation, reasoning, and agentic workflows.
• Oversee the end-to-end data lifecycle from customer intake to delivery, including collaboration with the researchers for defining the data requirements, guidelines, designing the data annotation steps/workflow, quality metrics, review workflows, and delivery setup.
• Serve as the primary contact for large engagements; manage stakeholder expectations, requirements gathering, and delivery timelines.
• Collaborate with engineering, product, research, and delivery teams to ensure technical feasibility and alignment across workstreams.
• Develop, document, and refine best practices for prompt evaluation, data schema design, evaluation metrics (e.g., win rate, pairwise preference), and human-in-the-loop QA.
• Manage and mentor leads, program managers, and annotators working across multiple AI training pods.
• Operate as a strategic business partner to our customers; provide insight on tradeoffs, resourcing, and performance metrics.
• Build internal capability by harvesting reusable assets and contribute in continued improvements across data quality, tools and processes.

Required Qualifications:

• 10+ years of experience building and leading large scale technical delivery teams. Proven ability to lead large cross functional teams (100+) by building strong operations
• Bachelor s degree in Engineering, Computer Science or equivalent practical experience leading large-scale technical initiatives.
• Demonstrated experience managing large-scale dataset generation or annotation for LLMs, ideally with experience in RLHF or SFT pipelines.
• Strong understanding of quality review mechanisms including prompt win rate, agreement metrics, inter-annotator consistency, and preference modeling.
• Experience with human data generation across at least one of the modalities (Text, Image, Video, Audio)
• Technical fluency in data platforms, machine learning concepts and modern Machine Learning tooling (e.g., HuggingFace, LangChain, Weights & Biases).
• Strong communication skills with experience presenting to executive stakeholders, managing client escalations, and aligning delivery with strategic goals.

Nice-to-have Qualifications:

• Prior experience at AI Data Platform companies
• Past experience with retrieval augmented generation (RAG), fine-tuning, and human evaluation workflows.
• Familiarity with fine-tuning LLMs, prompt engineering at scale, and instruction dataset design.
• Technical fluency in Python and cloud infrastructure
• Familiarity with bench marks (SWE Bench, MMMLU, or equivalent)