Data Annotation Engineer

Louisville, Kentucky (Day 1 onsite), KY

Instant Apply

30+ days ago

Job Description

Data Annotation Engineer
Location - Louisville, Kentucky (Day 1 onsite)
Pay Rate: Market- based on experience

We are looking for an Annotation Program Lead to manage our AI Quality Annotation Program. This individual will oversee the operational execution of human review activities that establish the gold-standard datasets used to measure conversational AI performance, safety, and compliance.
The ideal candidate combines strong program management skills with experience in quality assurance, data annotation operations, and stakeholder engagement.

Lead the day-to-day operation of the AI annotation program.
Manage a team of annotators responsible for reviewing customer conversations and AI interactions.
Develop annotation guidelines, procedures, and quality standards.
Establish calibration programs and quality assurance processes to ensure consistency across reviewers.
Partner with Data Scientists to create and maintain gold-standard datasets.
Monitor annotation accuracy, throughput, and quality metrics.
Coordinate reporting and deliverables for Legal, Compliance, Responsible AI, and executive stakeholders.
Manage annotation workflows and tooling including setting up annotation jobs, running data processing scripts, and testing annotation UIs for quality before job launch.
Plan capacity requirements to support seasonal increases in review volume.
Support future expansion into multilingual evaluation programs.
Identify process improvements that increase efficiency and annotation quality.

Bachelor's degree or equivalent experience in Linguistics/Psychology.
5+ years of experience in program management, operations, quality assurance, data labeling, or related fields.
Experience leading teams and managing operational workflows.
Strong organizational and stakeholder management skills.
Ability to analyze quality metrics and drive continuous improvement initiatives.
Excellent written and verbal communication skills.
Familiarity with Python/R, SQL and Excel. Should be familiar with running scripts on language data.
Strong organizational and stakeholder management skills.
Ability to analyze quality metrics and drive continuous improvement initiatives.
Excellent written and verbal communication skills.
Familiarity with Python/R, SQL and Excel. Should be familiar with running scripts on language data.

Experience supporting AI, machine learning, or data annotation programs.
Familiarity with Responsible AI, compliance, legal review, or governance processes.
Experience developing operational standards and quality frameworks.
Experience managing vendor or contractor resources.

Numbers & Facts

Location

Louisville, Kentucky (Day 1 onsite), KY

Skills

Analysis Skillsunmatched
Artificial Intelligence (AI)unmatched
Artificial Intelligence (AI) Programming Languagesunmatched
Calibrationunmatched
Capacity Requirements Planning (CRP)unmatched
Communication Skillsunmatched
Continuous Improvementunmatched
Data Processingunmatched
Data Scienceunmatched
Data Setsunmatched
Leadershipunmatched
Legalunmatched
Linguisticsunmatched
Machine Learningunmatched
Machine Toolunmatched
Multilingualunmatched
Operations Managementunmatched
Operations Planningunmatched
Organizational Skillsunmatched
Presentation/Verbal Skillsunmatched
Process Developmentunmatched
Process Improvementunmatched
Project/Program Managementunmatched
Psychologyunmatched
Python Programming/Scripting Languageunmatched
Quality Assuranceunmatched
Quality Assurance Methodologyunmatched
Quality Managementunmatched
Quality Metricsunmatched
R Programming Languageunmatched
SQL (Structured Query Language)unmatched
Safety Complianceunmatched
Scripting (Scripting Languages)unmatched
Software Testingunmatched
Standards Developmentunmatched
Team Lead/Managerunmatched
Test Scriptsunmatched
Training Data Setsunmatched
Vendor/Supplier Managementunmatched
Writing Skillsunmatched

Level up your application

Professional resume templates

Browse dozens of recruiter approved resume templates, layouts and formats. Choose your favorite and make it your own in minutes.

Free resume templates

Resume Builder

Improve your existing resume or start from scratch and create a standout, ATS-friendly resume. Add job-specific content, download and apply.

Resume Builder