Senior Manager, Data Quality & Evaluation

Accountabilities Design and manage quality frameworks for AI data and evaluation programs, including quality standards, acceptance criteria, review processes, and performance metrics. Translate customer requirements into scalable quality workflows that support data collection, annotation, evaluation, review, calibration, and reporting activities. Identify quality risks early and collaborate with delivery teams to resolve issues affecting timelines, customer confidence, or program outcomes. Build repeatable quality processes for calibration, QA sampling, adjudication, reviewer performance tracking, and customer reporting. Lead quality operations across multilingual evaluation, speech and audio review, transcription, annotation, human preference evaluation, expert review, coding evaluation, and AI model assessment programs. Develop and improve rubrics, task instructions, reviewer guidelines, calibration exercises, golden datasets, scorecards, and reporting templates. Monitor quality indicators such as reviewer agreement, error trends, performance metrics, and root causes of quality variation. Turn quality insights into actionable improvements across instructions, training programs, tooling, staffing models, and operational workflows. Support strategic customer programs through quality reviews, business discussions, escalations, retrospectives, and improvement plans. Partner with internal teams to ensure programs are designed for quality success from initial planning through production delivery. Build reusable quality assets and standardized approaches that improve scalability across multiple programs. Lead, coach, and develop Quality Managers, Quality Leads, Quality Specialists, reviewers, and other contributors involved in data quality operations. Promote a culture of quality ownership, accountability, continuous improvement, and operational excellence. Requirements 5+ years of experience in quality operations, data operations, AI data services, localization quality, annotation quality, evaluation operations, trust and safety quality, or a related field. Proven experience managing quality programs for complex customer accounts or large-scale operational delivery environments. Strong understanding of QA methodologies, calibration processes, sampling strategies, adjudication, error analysis, and performance reporting. Experience collaborating with cross-functional teams including operations, delivery, supply chain, sales, and customer-facing stakeholders. Strong analytical skills with the ability to transform quality data into operational improvements and strategic recommendations. Excellent written and verbal communication skills, with the ability to explain quality insights clearly to customers and senior stakeholders. Experience building processes in fast-moving, ambiguous environments where systems and workflows are continuously evolving. Strong leadership and people management skills, including coaching and developing quality specialists, reviewers, annotators, or operational teams. Experience with AI data workflows, RLHF, LLM evaluation, speech/audio evaluation, transcription, coding evaluation, multilingual evaluation, or expert review programs is highly valued. Experience designing evaluation rubrics, annotation guidelines, reviewer training materials, calibration workflows, or quality scorecards is a plus. Familiarity with human-in-the-loop data processes, annotation platforms, QA tools, dashboards, and distributed contributor networks is desirable. Knowledge of multilingual evaluation, cultural considerations, language quality risks, or specialized domain review processes is beneficial. Benefits Fully remote position with flexibility to work from anywhere. Opportunity to contribute to the development of AI systems through high-impact data quality initiatives. Leadership role with significant influence over quality strategy, processes, and operational standards. Collaboration with international teams working on advanced AI programs. Opportunity to build scalable systems and shape the future of AI evaluation operations. Professional growth opportunities within a fast-evolving technology environment. Dynamic workplace focused on innovation, continuous improvement, and operational excellence.