AI Operations Specialist - LLM & Conversation Systems

AI Operations Specialist – LLM & Conversation Systems Position Type: Full-Time, Remote Working Hours: U.S. Business Hours About the Role We are looking for a hands-on AI Operations Specialist to own and improve our production AI conversation systems. We are an AI-powered SMS lead recovery platform serving telehealth brands, processing millions of messages each month. Our AI systems interact directly with customers, and their performance directly impacts client conversions. In this role, you'll onboard new clients, configure AI conversation systems, monitor live conversations, diagnose issues, improve prompts and system behavior, and measure whether your changes actually improve conversion. This is not a theoretical AI role. We're looking for someone who has run LLM systems in production and understands how to continuously monitor, troubleshoot, test, and improve them. Responsibilities Client Onboarding & AI Setup Onboard new clients into the AI conversation platform. Set up webhook integrations and client knowledge bases. Configure campaigns and conversation workflows. Ensure new client systems are properly tested before going live. Troubleshoot integration and configuration issues during onboarding. Production AI Operations Monitor live AI-generated conversations and customer interactions. Review conversations to identify errors, inconsistencies, and missed opportunities. Diagnose AI failure modes and determine their root causes. Identify issues affecting conversatio