Quality Lead - AI & Automation Auditing (BPO Contact Center)

AI QA Framework & Testing

Operationalize and execute client-approved QA frameworks, evaluation rubrics, and test scripts tailored for AI chatbots, virtual agents, and copilot tools.

Perform routine pre-release User Acceptance Testing (UAT) and regression testing across prompt deployments, Knowledge Base (KB) updates, and automated workflow releases within the operational environment.

Operational Auditing & Performance Monitoring

Conduct daily audits of live AI-driven conversations, automated resolutions, and self-service interactions to detect intent misclassifications, system hallucinations, incorrect policy applications, or journey drops.

Audit the handover bridge between AI and Human-Assist agents to evaluate context retention, CRM data pass-through accuracy, and agent handle times post-transfer.

Closed-Loop Vendor Feedback & Defect Analysis

Perform root-cause defect analysis on resolution failures, escalation spikes, and recurring customer friction points.

Synthesize audit data into actionable weekly reports; feed continuous improvement insights to Client AI Automation Leads, KB Managers, and Product Engineering for system retuning and sprint updates.

Identify out-of-scope traveler queries and KB content gaps, recommending new intents and articles to safely broaden containment coverage.

Operational Metrics & SLA Compliance

Measure, track, and report on primary operational and AI quality KPIs:

Intent Recognition Accuracy

Resolution Precision & Answer Accuracy

Containment Rate & Defect Rate

Handover Context Retention Accuracy

Traveler Sentiment Score (TSS) / CSAT Impact

Hold operational teams and frontline agents accountable for adhering to updated AI copilot workflows and knowledge standards.

Experience

Contact Center QA Background: 2+ years of experience in quality management, operational auditing, or process improvement within a high-volume BPO/contact center environment.

AI & Automation Auditing: Demonstrated experience testing, auditing, or evaluating AI chatbots, conversational flows, copilot integrations, or workflow automation tools.

Travel Industry (Preferred): Prior experience in travel, hospitality, or tech-enabled customer support operations is strongly preferred.

Methodology: Familiarity with operational process improvement tools (Lean, Six Sigma, Root Cause Analysis / 5 Whys).

Core Competencies & Skills

Conversational AI Fluency: Deep understanding of natural language processing (NLP/NLU) behavior, intent mapping structures, and failure patterns (e.g., hallucination, drift, misclassification).

Analytical & Data-Driven: Ability to turn transcript audits, CSS feedback, and performance dashboard data into actionable insights, using AI tools to support analysis and validating outputs for accuracy and relevance.

Cross-Functional Communication: Clear written and verbal reporting skills to present quality defects and technical feedback to both BPO leadership and client technical teams.

Quality Standard-Setting: Meticulous attention to detail with an unyielding commitment to brand trust and effortless traveler resolution.

All your information will be kept confidential according to EEO guidelines.

We are seeking an operational Knowledge AI Quality Lead from our BPO partner to oversee, monitor, and elevate the performance of our AI-driven customer service tools and hybrid support workflows. Positioned within the partner's contact center operations, this role ensures that automated workflows, virtual assistants, agent copilots, and human-assist handovers deliver reliable, accurate, and compliant customer experiences.

The ideal candidate combines operational contact center auditing expertise with a deep understanding of AI conversational dynamics, acting as the frontline bridge between real-time vendor performance data and client-side engineering, product, and knowledge teams.