WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=vljxQZfJ9wY NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/vljxQZfJ9wY.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:03.320 --> 00:00:04.520 Hey everyone. s2 00:00:04.520 --> 00:00:17.360 My name is Nishant Gupta and I'm a software engineering tech lead at Meta working on building the training and inference infrastructure for the Meta Super Dangerous Lab and the infrastructure organization. s3 00:00:17.360 --> 00:00:21.920 Today we're going to be talking about production of ads for agentic systems. s4 00:00:21.920 --> 00:00:25.440 When most people hear the word evaluation, they think about benchmarks. s5 00:00:25.440 --> 00:00:28.080 A model scores 90% on a benchmark. s6 00:00:28.080 --> 00:00:30.280 A new version scores 92%. s7 00:00:30.280 --> 00:00:31.800 The team celebrates. s8 00:00:31.800 --> 00:00:35.760 But agentic systems have fundamentally changed what the evaluation means. s9 00:00:35.760 --> 00:00:38.760 Today the systems don't simply generate answers. s10 00:00:38.760 --> 00:00:39.720 They plan. s11 00:00:39.720 --> 00:00:40.480 They call tools. s12 00:00:40.480 --> 00:00:41.520 They retrieve information. s13 00:00:41.520 --> 00:00:42.840 They execute workflows. s14 00:00:42.840 --> 00:00:45.720 They interact with the production infrastructure. s15 00:00:45.720 --> 00:00:49.160 The question is no longer did the model generate the right answer? s16 00:00:49.160 --> 00:00:51.720 The question is did the system behave correctly? s17 00:00:51.720 --> 00:00:59.680 Today I would like to discuss how evaluation is evolving from model benchmarking into production infrastructure. s18 00:01:02.360 --> 00:01:06.360 This is the problem almost every AI organization is encountering today. s19 00:01:06.360 --> 00:01:09.000 Offline benchmarks continue improving. s20 00:01:09.000 --> 00:01:12.000 Yet production reliability often remains unpredictable. s21 00:01:12.000 --> 00:01:13.200 Why is that? s22 00:01:13.200 --> 00:01:15.720 Because benchmarks measure model capability. s23 00:01:15.720 --> 00:01:17.920 Production measures system behavior. s24 00:01:17.920 --> 00:01:25.400 A benchmark doesn't capture tool failure, API outage, context changes, user variability, long running workflows. s25 00:01:25.400 --> 00:01:30.760 And as systems become more autonomous, the gap between the benchmark performance and production performance grows. s26 00:01:30.760 --> 00:01:34.200 The result is what many teams experience today. s27 00:01:34.200 --> 00:01:40.120 High benchmark scores, as you can see, but unreliable production behavior. s28 00:01:41.560 --> 00:01:44.800 Traditional LLM evaluation focus on outputs. s29 00:01:44.800 --> 00:01:48.160 But we should ask the question did the model produce the correct answer? s30 00:01:48.160 --> 00:01:51.320 Agentic systems force us to ask a different question. s31 00:01:51.320 --> 00:01:52.960 Did the system behave correctly? s32 00:01:52.960 --> 00:02:01.360 Behavior includes planning quality, tool usage, execution, workflow execution, recovery from failures, decision-making. s33 00:02:01.360 --> 00:02:06.200 In other words, we are moving from evaluating answers to evaluating workflows. s34 00:02:06.200 --> 00:02:10.679 And that requires fundamentally different evaluation architectures. s35 00:02:10.800 --> 00:02:15.040 Many teams still think hallucinations are the primary AI failure modes. s36 00:02:15.040 --> 00:02:17.720 In production, they are often just one category. s37 00:02:17.720 --> 00:02:21.959 Agentic systems introduce an entire hierarchy of failure modes. s38 00:02:21.959 --> 00:02:26.480 At the very foundation, the memory failures, retrieval failures, safety failures. s39 00:02:26.480 --> 00:02:31.840 As you go up, you have to think about reasoning mistakes, poor planning, incorrect tool execution. s40 00:02:31.840 --> 00:02:35.680 At the highest layer, you have to think about multi-agent coordination failures. s41 00:02:35.680 --> 00:02:42.640 And this is why evaluating only model output misses the most production risk we observe. s42 00:02:43.000 --> 00:02:50.920 One of the most useful mindset shifts is to stop thinking like researchers and start thinking like a SRE or a production engineer. s43 00:02:50.920 --> 00:02:53.440 SREs don't measure success using accuracy. s44 00:02:53.440 --> 00:02:57.040 They measure reliability, availability, latency, cost, recovery. s45 00:02:57.040 --> 00:02:59.480 And agentic systems require the same approach. s46 00:02:59.480 --> 00:03:02.120 The goal is not maximizing the benchmark scores. s47 00:03:02.120 --> 00:03:05.160 The goal is to maximize dependable outcomes. s48 00:03:05.160 --> 00:03:07.720 Reliability becomes the North Star metric. s49 00:03:07.720 --> 00:03:11.760 Accuracy becomes the only input. s50 00:03:14.760 --> 00:03:18.959 In this pyramid is how I think personally think about modern AI evaluation systems. s51 00:03:18.959 --> 00:03:21.120 At the bottom, you can see they are benchmarks. s52 00:03:21.120 --> 00:03:21.840 They are useful. s53 00:03:21.840 --> 00:03:22.440 They are scalable. s54 00:03:22.440 --> 00:03:23.280 They are repeatable. s55 00:03:23.280 --> 00:03:25.320 But their operational value is limited. s56 00:03:25.320 --> 00:03:28.160 In the middle, there are scenario-based evaluations. s57 00:03:28.160 --> 00:03:30.519 These simulate realistic workflows. s58 00:03:30.519 --> 00:03:33.160 And at the very top, you see production telemetry. s59 00:03:33.160 --> 00:03:36.160 This is where the highest value evaluation signals come from. s60 00:03:36.160 --> 00:03:43.720 The surprising insight is that the most evaluation data often comes from real users interacting with real systems. s61 00:03:45.120 --> 00:03:46.920 Now, let's talk about AI offline evaluations. s62 00:03:46.920 --> 00:03:50.360 So, offline evaluation still matters, but the methodology changes. s63 00:03:50.360 --> 00:03:52.840 Instead of evaluating prompts, we evaluate scenarios. s64 00:03:52.840 --> 00:03:57.280 For example, a customer support workflow, a code generation workflow, a research workflow. s65 00:03:57.280 --> 00:03:59.680 The agent operates inside that simulated environment. s66 00:03:59.680 --> 00:04:08.920 We measure the task completion rate, tool correctness, planning quality, resource usage, which is which becomes exponentially high at high scale. s67 00:04:08.920 --> 00:04:13.959 The key takeaway, agent evaluation should be scenario driven, not prompt driven. s68 00:04:14.400 --> 00:04:18.480 Once a system reaches production, every interaction becomes a signal. s69 00:04:18.480 --> 00:04:21.359 This is one of the biggest shifts in evaluation thinking. s70 00:04:21.359 --> 00:04:23.800 Production traffic is no longer just traffic. s71 00:04:23.800 --> 00:04:25.840 It becomes evaluation data. s72 00:04:25.840 --> 00:04:31.200 We collect execution traces, user outcomes, escalations, failures, feedback signals. s73 00:04:31.200 --> 00:04:38.200 Production is the largest and the most representative evaluation data any organization will ever have. s74 00:04:39.200 --> 00:04:42.440 Many organizations view humans as fallback systems. s75 00:04:42.440 --> 00:04:44.560 I think that's a wrong framing. s76 00:04:44.560 --> 00:04:46.280 Humans are the evaluators. s77 00:04:46.280 --> 00:04:48.840 They provide signals that automated systems cannot. s78 00:04:48.840 --> 00:04:52.040 They assess correctness, trust, usefulness, safety. s79 00:04:52.040 --> 00:04:59.200 These signals become really critical for calibrating evaluation pipelines and identifying blind spots in automated metrics. s80 00:04:59.200 --> 00:05:05.080 The most successful systems combine automated evaluation with targeted human review. s81 00:05:06.280 --> 00:05:08.680 Now, agent systems drift constantly. s82 00:05:08.680 --> 00:05:09.840 Model changes. s83 00:05:09.840 --> 00:05:13.360 You have a new version every couple of weeks or months. s84 00:05:13.360 --> 00:05:17.160 Prompts can change, tools can change, user behavior can change. s85 00:05:17.160 --> 00:05:20.520 The challenge is that no longer single change appear catastrophic. s86 00:05:20.520 --> 00:05:22.440 Reliability slowly degrades. s87 00:05:22.440 --> 00:05:26.480 Success rate declines, escalation increases, tool failure rises. s88 00:05:26.480 --> 00:05:31.160 Without continuous evaluation, teams often don't discover drift until users complain. s89 00:05:31.160 --> 00:05:34.600 Continuous monitoring becomes essential. s90 00:05:36.160 --> 00:05:40.120 Observability and evaluation are inseparable inseparable. s91 00:05:40.120 --> 00:05:49.400 To evaluate an agent, we need visibility into the reasoning paths, the tool calls, the memory access, execution timelines, the state transitions, as you can see here in this chart, s92 00:05:49.400 --> 00:05:51.000 traditional logs are not sufficient. s93 00:05:51.000 --> 00:06:00.960 We need detailed traces, just like with any deep nested microservice architecture for any application or service you're talking about. s94 00:06:00.960 --> 00:06:03.960 Agent traces become the equivalent of distributed tracing for autonomous workloads. s95 00:06:03.960 --> 00:06:08.280 Without observability, evaluation becomes the guesswork. s96 00:06:10.760 --> 00:06:17.280 Let's talk about the continuous evaluation loop because evaluation is an always-running service, not a testing phase. s97 00:06:17.280 --> 00:06:31.080 Historically, evaluation always happened before deployment, but now evaluation continues after deployment till you may identify issues, as you can see in A. Human reviews the edge cases, feedback improves the data sets, offline scenarios validate updates, s98 00:06:31.080 --> 00:06:32.640 the loop never stops. s99 00:06:32.640 --> 00:06:38.040 Evaluation is no longer just a phase, it's an operational capability. s100 00:06:38.480 --> 00:06:42.200 This is probably the most important slide in this presentation. s101 00:06:42.200 --> 00:06:45.760 Every metric shown here maps directly to a business outcome. s102 00:06:45.760 --> 00:06:47.800 Task completion measures value delivered. s103 00:06:47.800 --> 00:06:50.400 Tool success measures operational reliability. s104 00:06:50.400 --> 00:06:52.600 Escalation rate measures human burden. s105 00:06:52.600 --> 00:06:55.160 Safety violations measure risk exposure. s106 00:06:55.160 --> 00:06:57.160 Latency affects user experience. s107 00:06:57.160 --> 00:06:58.840 Cost determines scalability. s108 00:06:58.840 --> 00:07:02.160 Recovery rate reflects resilience. s109 00:07:02.160 --> 00:07:05.280 And notice but notice that accuracy is missing. s110 00:07:05.280 --> 00:07:11.960 It's not because accuracy doesn't matter, but because business success depends on much more than just accuracy. s111 00:07:12.600 --> 00:07:16.400 Now, this is the architecture where the industry is heading more or less. s112 00:07:16.400 --> 00:07:20.760 Evaluation becomes part of the control plane, not a separate tool, not an offline process. s113 00:07:20.760 --> 00:07:30.960 The control plane continuously, which observes the systems, collects telemetry, runs simulations, coordinates human review, and the execution plane performs the work. s114 00:07:30.960 --> 00:07:38.200 The control plane measures and governs the behavior, and this separation is becoming a foundational pattern for production AI systems. s115 00:07:39.240 --> 00:07:40.840 Now, let's summarize the key lessons. s116 00:07:40.840 --> 00:07:45.800 First, benchmarks remains necessary, but they are insufficient. s117 00:07:45.800 --> 00:07:50.520 Second, agent systems must be valued as workflows, not individual outputs. s118 00:07:50.520 --> 00:07:55.240 Third, production telemetry is a most important evaluation signal. s119 00:07:55.240 --> 00:07:58.960 Fourth, reliability ultimately matters more than raw model accuracy. s120 00:07:58.960 --> 00:08:06.160 And finally, evaluation is becoming the infrastructure, not testing, not QA infrastructure. s121 00:08:06.160 --> 00:08:11.720 This is the shift every organization building agent AI will eventually need to make.