About The RoleSupport human-in-the-loop AI training workflows for large language models and multimodal systems. You will improve training data quality by completing data labeling, RLHF preference ranking, prompt evaluation, and QA evaluation tasks using structured annotation tools and written guidelines.Key ResponsibilitiesCreate and review labeled datasets for NLP, computer vision, and multimodal use casesPerform RLHF preference ranking and rubric-based scoringRun prompt and response evaluations for helpfulness, factuality, and safetyExecute content safety labeling (policy, harassment, self-harm, sensitive categories)Follow annotation guidelines and document rationales for edge casesConduct QA evaluation via sampling plans, audits, and error taxonomy trackingParticipate in calibration sessions to reduce rater variance and improve agreementTrack throughput and quality metrics that impact LLM training pipelinesRequired QualificationsStrong written English and structured reasoningAbility to follow detailed annotation guidelines with consistent judgmentComfort using spreadsheets, web tools, or labeling interfacesFamiliarity with model evaluation and prompt/response patternsAttention to detail and evidence-based QA evaluation approachComfort handling sensitive content as part of content safety labelingWork DetailsRemote, full-time work delivered through online tooling and written annotation guidelines. Quality is measured via audits, disagreement rates, rubric adherence, and training data quality outcomes.CompensationCompetitive hourly rate: $30–$50/hr (USD).#J-18808-Ljbffr