{ "video": { "id": "obfPzbapMIM", "title": "Evals for High-Stakes AI Agents — Doneyli De Jesus & Annabell Schäfer, Langfuse", "duration": 3212, "upload_date": null, "channel": "AI Engineer", "source": "AI Engineer", "event_format": "" }, "analysis": { "video_id": "obfPzbapMIM", "title": "Evals for High-Stakes AI Agents — Doneyli De Jesus & Annabell Schäfer, Langfuse", "one_liner": "In high-stakes, regulated settings, build a ground-truth dataset with domain experts first, then gate every agent config change (prompt, model, tools) through a CI experiment that blends LLM-as-a-judge evaluators with deterministic ones, because agents can fail while system metrics look green.", "summary": "Langfuse's Annabell Schäfer and Doneyli De Jesus present the 'AI engineering loop': tracing, monitoring, datasets, experiments and evals, repeated continuously with production data fed back into datasets. Low-stakes internal agents can ship with tracing and monitors first. Customer-facing and regulated use cases need a ground-truth dataset first, built through expert curation, shadow runs, or a human-in-the-loop system that is gradually sliced into automatable parts. Doneyli demos a fictional banking assistant that covers PII masking, guardrail scores for investment advice, multimodal call-centre traces, annotation queues, LLM-derived failure-mode categories, and a CI eval gate triggered by promoting a prompt version. The team also shows business dashboards built from the same scores.", "key_points": [ "Example of a silent failure: a support agent tells the user it issued a refund, but the refund tool was never called. The trace is green with no latency issues, and the problem only shows up weeks later when the user complains.", "The AI engineering loop: tracing (an inspectable log for humans and coding agents) → monitoring (aggregate cost, latency and quality signals, e.g. users writing in all caps) → datasets (representative inputs plus expected outputs, behaviours or keywords, including out-of-scope requests) → experiments (change one variable at a time: prompt, model, context or tool description) → evals (LLM-as-a-judge plus code-based checks).", "Three ways regulated teams build ground truth: expert curation, shadow runs (the manual process continues while the agent runs in parallel for comparison), and a full human-in-the-loop system later sliced into confidently automatable parts and parts that always need a human.", "Banking demo: PII such as a card number typed in clear text is masked before it is persisted to Langfuse. Input guardrail scores flag intents like 'can I put my savings in bitcoin' as investment advice. Speech-to-text call recordings are traced with links to the audio and the transcript.", "Error analysis workflow: sample 10–20% of production traces (filtered, e.g. to disputes) into an annotation queue. Domain experts give simple scores plus free-text comments, at roughly 30 seconds per trace (an hour of their time). An LLM then clusters the comments into failure-mode categories, which become the evaluator dimensions (e.g. source correctness, formal register, language match, no unsolicited upsell).", "Blend LLM-as-a-judge evaluators with deterministic code evaluators (e.g. regex or exact match). Judges have a nonzero error rate, while code checks give the same output 100% of the time for the same input.", "CI/CD demo: a Langfuse automation fires on a prompt change, such as promoting version 4 to the 'production' label. It launches a GitHub CI job that runs the experiment on the dataset and returns a pass/fail gate with a minimum threshold per score. Prompts can be versioned in Langfuse or in GitHub/GitLab.", "Business dashboard: 1,400 interactions in 7 days needed no human agent, at a cost under $7, compared with a $15–20/hour contact-centre worker. It also shows top failure modes (out of scope, informal register, dispute not resolved), an 18% thumbs-down rate, and 89 investment-advice flags, each of which can be filtered into a saved trace view.", "Q&A: Langfuse started on Postgres and moved to ClickHouse for scale. Data can be exported to S3 or accessed via API, CLI or MCP. It can be self-hosted on AWS. To align an LLM judge, have a human label a representative sample, optimise the judge against those labels, re-check over time, and start with too many false positives, then tune them down." ], "takeaways": [ "Before shipping a customer-facing agent in a regulated domain, write down hard limits (e.g. never give investment advice) and put them in your eval set, so you at least know when they are breached.", "Make annotation cheap for domain experts: a couple of scores plus raw comments, possibly dictated. Let an LLM derive the failure-mode categories instead of asking experts to invent them.", "Wire evals into CI so every prompt, model or tool change runs against the dataset with per-score minimum thresholds. That gives you an evidence trail instead of shipping 'on vibes'.", "Don't just tell a coding agent 'fix my agent, make no mistakes'. Automate instrumentation, failure-mode finding and running many experiments, but keep your own taste and your domain experts in charge of curating the datasets.", "To get started, install the Langfuse agent skill and collect ground truth from historical data, production traces, or domain-expert samples." ], "topics": [ "evals", "agents", "observability", "llm-as-a-judge", "regulated-industries", "datasets", "ci-cd", "human-in-the-loop" ], "tools": [ "Langfuse", "Langfuse skill", "Langfuse Academy", "OpenAI", "Anthropic", "ChatGPT", "Claude", "GitHub", "GitLab", "ClickHouse", "Postgres", "Amazon S3", "Amazon EC2", "MCP", "Wispr Flow", "Y Combinator", "Ramp", "Canva", "Khan Academy" ], "quotes": [ { "text": "AI agents can fail while the system metrics themselves look green.", "at": "02:11", "url": "https://www.youtube.com/watch?v=obfPzbapMIM&t=131s" }, { "text": "if you rely 100% just on LLM as a judge there's a nonzero chance that some of the outputs are going are not going to be accurate because that's just the nature of the LLMs.", "at": "31:24", "url": "https://www.youtube.com/watch?v=obfPzbapMIM&t=1884s" }, { "text": "You're just telling them, give me your unfiltered raw commentary on these traces and then you're going to use an LLM to then figure out the categories because that's what they are great at.", "at": "28:39", "url": "https://www.youtube.com/watch?v=obfPzbapMIM&t=1719s" }, { "text": "Please fix my agent. Make no mistakes.", "at": "46:05", "url": "https://www.youtube.com/watch?v=obfPzbapMIM&t=2765s" } ], "words": 11459 }, "summary_url": "/#obfPzbapMIM", "transcript": { "html": "/transcripts/obfPzbapMIM.html", "txt": "/transcripts/obfPzbapMIM.txt", "vtt": "/transcripts/obfPzbapMIM.vtt" } }