{ "video": { "id": "pyvRID_CZZU", "title": "We Built an AI Support Agent That Resolves 80% of Tickets — AssemblyAI", "duration": 979, "upload_date": null, "channel": "AI Engineer", "source": "AI Engineer", "event_format": "" }, "analysis": { "video_id": "pyvRID_CZZU", "title": "We Built an AI Support Agent That Resolves 80% of Tickets — AssemblyAI", "one_liner": "AssemblyAI replaced an off-the-shelf docs bot that resolved 10% of tickets with Joey, their own support agent built on the Claude Agent SDK. In its first week, with what they called a pretty naive implementation, Joey resolved 80% of all inbound tickets end to end for about $700 a month.", "summary": "Matt Lawler, a forward deployed engineer (FDE) at AssemblyAI, says FDEs should try to automate themselves out of a job, because hands-on FDE work can't keep up with about 1,000 API signups a day. A vendor support bot resolved only 10% of conversations, and the team couldn't change its system prompt, tools or RAG setup. So they built Joey on the Claude Agent SDK, with local markdown docs, Voyage embeddings, agentic file-system search and Railway deploys. Joey reached 80% end-to-end resolution, then got a voice mode through AssemblyAI's own Voice Agent API, which let the team dogfood the product their customers build.", "key_points": [ "The off-the-shelf support bot pointed at the docs resolved only about 10% of conversations. With around 1,000 conversations a day, that left about 900 tickets a day for the team. They also couldn't iterate: they had no access to the system prompt, tools or RAG setup, and the vendor's answer was 'it's on the road map.'", "Joey is the first line of defence for every inbound channel: the site chat widget, contact sales and the support email alias. Customers can't reach a human without going through Joey and asking him to escalate. So the 80% figure covers 100% of inbound tickets, not a hand-picked subset.", "The architecture has four parts. (1) The docs, changelog and pricing pages are checked out as local markdown and stay in sync with every docs update; each page keeps its URL so Joey can cite sources. (2) Voyage embeddings retrieve the most relevant docs up front. (3) Joey can search his own file system agentically when he needs to combine several sources. (4) Everything is deployed on Railway as a Docker container.", "Behaviour is steered by what Lawler described as 'like a 30,000 line' CLAUDE.md of guardrails and advice. When Joey gives a wrong answer or a customer has a bad experience, the team ships an update to this file.", "A pull request goes live on Railway within about 30 seconds. The team has watched a live conversation, caught a bug and deployed a fix during that same session without the customer noticing.", "The 20% that still escalates are cases that need a human: rate changes, data opt-outs, signing agreements, or anything involving an FDE or legal. That list doubles as the automation backlog. Joey can now handle BAAs (business associate agreements) and quote and negotiate prices from your expected usage hours.", "Joey has a voice mode built on AssemblyAI's Voice Agent API. It runs speech in and speech out over one websocket, chaining speech-to-text, an LLM and text-to-speech, and it handles pauses, interruptions and barge-in. It uses the same tools as the text version, and a phone number was planned for the same week.", "In a live demo, a user building an ambient medical scribe asked about a BAA. Joey said they could execute it themselves but would need a paid account with a card on file, and that signing it automatically opts them out of model training. He linked the relevant docs." ], "takeaways": [ "If you're an FDE, turn what you know into automation, for example a support agent. Don't be the bottleneck on customer experience, and don't count on hiring more FDEs to keep up.", "Build your support agent on infrastructure you control: your own prompt, tools, retrieval and deploys. A vendor bot that you can't change is hard to improve.", "Give the agent your docs as local markdown that syncs on every docs update and keeps source URLs. Combine embedding retrieval with agentic file-system search.", "Set up a fast fix loop: monitor conversations, patch the CLAUDE.md guardrails and redeploy in seconds. Treat escalation reasons as your list of what to automate next.", "Dogfood your own product by building what your customers build. Running into the same problems, such as latency and turn-taking, gives you more empathy and lets you give better advice." ], "topics": [ "agents", "customer-support", "voice-agents", "forward-deployed-engineering", "rag", "dogfooding", "deployment" ], "tools": [ "AssemblyAI", "AssemblyAI Voice Agent API", "Claude Agent SDK", "Claude", "CLAUDE.md", "Pylon", "Voyage", "Railway", "Docker", "Fireflies", "Slack", "AWS EC2" ], "quotes": [ { "text": "try to automate yourself out of a job. I think that should be your role every single day.", "at": "04:00", "url": "https://www.youtube.com/watch?v=pyvRID_CZZU&t=240s" }, { "text": "Well, it's on the road map.", "at": "05:12", "url": "https://www.youtube.com/watch?v=pyvRID_CZZU&t=312s" }, { "text": "if our entire doc site was down right now, you could go to Joey and still get live answers on all the newest stuff that we've shipped", "at": "06:45", "url": "https://www.youtube.com/watch?v=pyvRID_CZZU&t=405s" }, { "text": "the best way to understand what your customers are building is to actually build their same product yourself.", "at": "10:10", "url": "https://www.youtube.com/watch?v=pyvRID_CZZU&t=610s" } ], "words": 4222 }, "summary_url": "/#pyvRID_CZZU", "transcript": { "html": "/transcripts/pyvRID_CZZU.html", "txt": "/transcripts/pyvRID_CZZU.txt", "vtt": "/transcripts/pyvRID_CZZU.vtt" } }