Manager, AI Analytics

Overview National Debt Relief is seeking a data-savvy Manager of AI Analytics to serve as an individual-contributor analytics expert supporting the measurement, testing, and optimization of agent-replacement AI across voice and chat channels. This role will execute the analytics framework that governs AI interaction performance by designing tests, tracking KPIs, and building the dashboards that give the business real-time visibility into how AI is performing against quality, containment, and client experience goals. Reporting to the Director of Digital & AI Analytics, this individual will be a hands-on analytical partner to AI Analytics and Engineering teams, translating complex model and interaction data into clear, actionable recommendations that inform the pace and scope of AI scale-up. Responsibilities Execute the measurement framework for AI Analytics, tracking performance of agent-replacement AI interactions across voice and chat channels. Design and run test analysis for AI model and prompt experiments (e.g., model version A/B tests, containment vs. escalation testing), translating results into go/no-go recommendations. Build and maintain dashboards and scorecards tracking AI-handled interaction KPIs — containment rate, accuracy, quality/CSAT, and escalation rate. Partner with AI Analytics and Engineering teams to ensure AI interaction data (transcripts, model outputs, outcomes) flows cleanly into unified reporting. Analyze AI-handled interaction volume and perform