WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=7WUzHIF32vA NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/7WUzHIF32vA.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:12.480 --> 00:00:17.760 I think why all of us are here primarily is because evals are currently very broken, right? s2 00:00:17.760 --> 00:00:21.720 And like that is what we're trying to explore and understand how to go about it. s3 00:00:21.720 --> 00:00:28.880 More often than not, I think evals are broken because we overcomplicate it and we don't really understand, right? s4 00:00:28.880 --> 00:00:35.800 We hear things like the eval is more important than the model and the eval is more important than the hardness or whatnot, but what even is an eval, right? s5 00:00:35.800 --> 00:00:43.600 It's a question that we all sort of come to and like this is something that I consistently hit when I speak with people building high-stake applications to even understand s6 00:00:43.600 --> 00:00:46.640 to the first principle level of what an eval is, right? s7 00:00:46.640 --> 00:00:51.840 And what is an eval is just a abbreviation of the term evaluation, right? s8 00:00:51.840 --> 00:01:00.480 And how we understand evaluation is like we know exams, which are examinations and this is a concept that we understand in human beings, right? s9 00:01:00.480 --> 00:01:09.160 And if you think about it, as people we write a variety of exams through our life, like we write the SATs, the GMAT, the bar exam and like regardless of what we do. s10 00:01:09.160 --> 00:01:17.480 That is the same sort of idea you can apply as a parallel into evals and then you can think about evals as those examinations for AI systems, right? s11 00:01:17.480 --> 00:01:21.600 You have the Sweet Bench, the humanities law exam and all of this. s12 00:01:21.600 --> 00:01:30.160 In multiple ways, this is trying to evaluate and try to understand how a model or an agent will be working in something, right? s13 00:01:30.160 --> 00:01:40.960 At the end of the day, examinations and evals are just tasks, standards, and evidences to sort of evaluate and understand how something is to perform on something as you expect, right? s14 00:01:40.960 --> 00:01:44.160 And I think that brings us to a very key distinction here as well. s15 00:01:44.160 --> 00:01:49.960 An exam must answer a bounded question and let's take the example of the SATs. s16 00:01:49.960 --> 00:02:00.320 Um it's a college entrance exam and it tells you how good a student is on language and math, but it's just one signal of a college readiness, right? s17 00:02:00.320 --> 00:02:05.840 I come from India and even today in India your college admissions are done with a single examination. s18 00:02:05.840 --> 00:02:11.000 However, let's say you look at the US exam, SAT is one signal of college readiness, right? s19 00:02:11.000 --> 00:02:12.600 And I think that's extremely key. s20 00:02:12.600 --> 00:02:22.040 You should not index on one eval or one test and it should just be one signal of your readiness to the platform rather than trying to sort of over index on s21 00:02:22.040 --> 00:02:26.240 your model being really good at humanities class exam or at W bench, right? s22 00:02:26.240 --> 00:02:36.640 So, no eval or no test is a measure of someone's complete abilities and that should always be clear going into that. s23 00:02:36.640 --> 00:02:47.800 So, at the end of the day, what we try to answer with evals is is our performance matching the promise that we're making to the consumer or the user at any given point in time, right? s24 00:02:47.800 --> 00:02:58.120 And that's the job of any eval to make sure that we go ahead and ensure that the promise of the product of the agent that we're providing is meeting its performance to the consumer. s25 00:02:58.120 --> 00:03:01.360 Before we move ahead, I wanted to sort of give a small brief on who I am. s26 00:03:01.360 --> 00:03:02.680 I'm Ramakrishnan. s27 00:03:02.680 --> 00:03:06.000 I come from the city called Cochin in in South of India. s28 00:03:06.000 --> 00:03:11.040 I started coding very early on and started building machine learning classifiers in high school and whatnot. s29 00:03:11.040 --> 00:03:21.480 And I started my career with an Indian EdTech unicorn which gave me the opportunity in '21 to get a demo of a GPT-3 based product on how to better our content flow s30 00:03:21.480 --> 00:03:25.840 which changed my mind being someone who really loved machine learning from an early age. s31 00:03:25.840 --> 00:03:28.200 And I was like, this is going to really change the world, right? s32 00:03:28.200 --> 00:03:40.320 And my claim to fame or like what I was right in doing was in September 2022, we decided that there is a new wave of applications that are going to be possible that are going to be built on the GPT-3 API, s33 00:03:40.320 --> 00:03:40.880 right? s34 00:03:40.880 --> 00:03:51.880 And about 2 months after we went full-time, OpenAI released released their research preview for GPT-3.5 called ChatGPT and like that literally changed life for us and it validated us and killed us as well, right? s35 00:03:51.880 --> 00:04:05.520 So, went through multiple pivots and then we eventually landed on a pet care AI assistant and we monetized it using e-commerce and recommendation and engines on top of it and we got acquired in India and that was the end of that journey, right? s36 00:04:05.520 --> 00:04:15.320 Uh after that, I've been super interested in understanding how these systems evolve and because we were super early on building on GPT-3, evals were critical for us to build. s37 00:04:15.320 --> 00:04:19.480 So, that took me to being a red teamer at for the Nematron 3 models. s38 00:04:19.480 --> 00:04:26.480 I work at the Lawrence Berkeley Lab right now on a AI safety research and also with the London Business School data science and AI initiative. s39 00:04:26.480 --> 00:04:33.280 Broadly to explore how to sort of look at the new technology that we are working on and then how we see and explore, right? s40 00:04:33.280 --> 00:04:37.600 So, I am not aligned or like, you know, with anybody here, so I'm not going to sponsor a vendor. s41 00:04:37.600 --> 00:04:42.160 It's just to understand how do you think about eval and like build these high-stake applications, right? s42 00:04:42.160 --> 00:04:51.760 And the consolidation of the conversations that I've had over time with multiple implementations that we have done with millions of users using our AI product, sorry. s43 00:04:51.760 --> 00:04:53.440 So, this is something I strongly believe. s44 00:04:53.440 --> 00:04:56.720 So, evals are not going to be relevant in a decade's time. s45 00:04:56.720 --> 00:05:03.840 Because if we are to believe that model capability is to keep improving, we will not have to have as much eval as we do. s46 00:05:03.840 --> 00:05:08.240 However, in the midterm, like production cannot wait. s47 00:05:08.240 --> 00:05:19.919 The problem is if you don't have the right amount of eval structure, you get reports like the MIT GenAI divide where 95% of organizations are not able to get any returns because you are not delivering the promise in production s48 00:05:19.919 --> 00:05:22.000 that you're trying to make with the LLM, right? s49 00:05:22.000 --> 00:05:28.880 And that is what we have to solve with evals today and in 10 years time, evals is just going to be a validation of this. s50 00:05:28.880 --> 00:05:34.760 But today, it is to make sure that in production you are delivering the promise that you're going ahead for, right? s51 00:05:34.760 --> 00:05:42.440 So, the first thing that you have to think about as you build evals for high-stake applications is to define the most ambitious success, right? s52 00:05:42.440 --> 00:05:44.120 Before designing any test. s53 00:05:44.120 --> 00:05:48.200 Like, assume in in of AGI, what is your AI agent going to do? s54 00:05:48.200 --> 00:05:51.160 Or what is the best outcome you can get out of it, right? s55 00:05:51.160 --> 00:05:57.200 And setting this goal really gives you the scope of what you can go ahead and design further, right? s56 00:05:57.200 --> 00:05:58.840 And you start with the job. s57 00:05:58.840 --> 00:06:02.400 And simple the first question that you ask is what must happen. s58 00:06:02.400 --> 00:06:04.920 And what must never happen? s59 00:06:04.920 --> 00:06:06.600 And when should it stop? s60 00:06:06.600 --> 00:06:15.680 And you can extend it into other WH questions and first principles to go ahead and think about what could be the most ambitious outcome that you can get out of it, right? s61 00:06:15.680 --> 00:06:19.160 And aim at the job you need and not the score you can get. s62 00:06:19.160 --> 00:06:23.240 If you have an eval that is scoring a 100 of a 100, your eval is wrong, right? s63 00:06:23.240 --> 00:06:25.840 And you're not ambitious enough and you're not testing enough, right? s64 00:06:25.840 --> 00:06:29.840 If you look at any model eval today, people are at like 85, 86 max. s65 00:06:29.840 --> 00:06:31.320 Nobody gets to 100. s66 00:06:31.320 --> 00:06:41.919 So, you have to make sure that the evaluation and your ambition is beyond what the model ability today is for you to make sure that you're putting it to the stress test and not making it easy, right? s67 00:06:41.919 --> 00:06:45.680 And always test the whole workflow, right? s68 00:06:45.680 --> 00:06:47.240 And not just the final answer. s69 00:06:47.240 --> 00:06:52.919 And this is the original picture, and my request would be only change the background, right? s70 00:06:52.919 --> 00:06:55.320 And I am just looking at the output. s71 00:06:55.320 --> 00:07:01.480 And the output has changed the background to a cream background, but it has two handles now, right? s72 00:07:01.480 --> 00:07:04.600 So, it has to be very clear that you have to always test the work. s73 00:07:04.600 --> 00:07:06.560 And this is a very simplified example of this. s74 00:07:06.560 --> 00:07:08.960 Let's say you have a 100 steps in your workflow. s75 00:07:08.960 --> 00:07:15.000 Each step has to be evaluated for what goes into that step and what comes out of that step and what is the expected outcome. s76 00:07:15.000 --> 00:07:20.320 You can't look at the start point and the end point and be like, "Okay, if you look at the start point and the end point, the background has changed." s77 00:07:20.320 --> 00:07:22.760 But is that the objective that you're trying to achieve, right? s78 00:07:22.760 --> 00:07:28.320 So, always for look at the whole workflow when you're approaching things, right? s79 00:07:28.320 --> 00:07:37.880 And the third thing, and I'm I'm going to talk a little bit about something I've been working on in this space as well, is one score is not the system, right? s80 00:07:37.880 --> 00:07:41.240 And the idea is to build a holistic evaluation system. s81 00:07:41.240 --> 00:07:54.120 When you started off with models in 2022 and 2023, you were measuring for coding ability or let's say you were measuring for writing ability, etc. But today you have agentic persistent systems that have to do multiple things, right? s82 00:07:54.120 --> 00:07:58.560 So you have to make sure that you're designing a holistic evaluation system at all points in time. s83 00:07:58.560 --> 00:08:04.200 And the first element of that is that automated checks and eval being part of your CI pipeline, right? s84 00:08:04.200 --> 00:08:11.440 And your eval questions have to be asked to your agent every time they are going through a production deployment process, right? s85 00:08:11.440 --> 00:08:19.600 So that being part of the automated checks makes sure that it's not like one eval score you are taking at the time you're deciding the model. s86 00:08:19.600 --> 00:08:23.440 Continuously human review and make sure that the results make sense. s87 00:08:23.440 --> 00:08:28.120 Don't rely on your agents to be the reviewer of your eval at any point always, right? s88 00:08:28.120 --> 00:08:32.599 You can fast-track your eval process with it, but make sure you're doing human review. s89 00:08:32.599 --> 00:08:36.360 And the third part is adversarial test, right? s90 00:08:36.360 --> 00:08:43.360 Where can you break the workflow and like put it into challenging situations like where you can have adversarial testing? s91 00:08:43.360 --> 00:08:53.680 And one thing I've been recently working on and would love to talk about is um the idea of how you can choose the test for the failures it can reveal, right? s92 00:08:53.680 --> 00:08:56.440 And the idea of a test that can fight back. s93 00:08:56.440 --> 00:09:02.720 Today you have a task on SW bench, the agent or the model is asked to go ahead and perform one task and give an output. s94 00:09:02.720 --> 00:09:06.400 But there is no continuous thinking or evaluation, right? s95 00:09:06.400 --> 00:09:17.120 And getting one right answer more and more is not the same as making a good sequence of decisions and you need agentic action and long-chain reasoning and all of these things. s96 00:09:17.120 --> 00:09:23.040 For the longest time the best measure of in human beings have been board games like chess and go. s97 00:09:23.040 --> 00:09:36.640 And even when you go ahead to um like like evaluation of like a model, AlphaGo is a very important step to AI development, but it never very specifically trained model for playing Go. s98 00:09:36.640 --> 00:09:45.240 Today's general-purpose LLMs we don't test on Go or chess, and that is what I'm trying to solve with an open-source repo called E-Slams, which is electronic slams, and we make s99 00:09:45.240 --> 00:09:46.920 AI play games, right? s100 00:09:46.920 --> 00:09:48.320 So, you can just install the package. s101 00:09:48.320 --> 00:09:55.560 It's a Python PyPi package, and you can make your agent and your model play games and see how it works. s102 00:09:55.560 --> 00:09:57.480 And how can you put it to test, right? s103 00:09:57.480 --> 00:10:00.280 You have You're replacing an agent with an improved model. s104 00:10:00.280 --> 00:10:05.200 You might want your old agent to play with the new agent on on a chess game of chess. s105 00:10:05.200 --> 00:10:15.920 And let's say I don't I'm not saying the winner of that match is a better model, but how they enter that match and play that match will tell you a lot more about how to sort of go ahead with that model. s106 00:10:15.920 --> 00:10:23.960 Because playing chess, you are changing your strategy, you have to get out of blunders, and move that you make can become liabilities for you in the past. s107 00:10:23.960 --> 00:10:30.240 So, how is your agent reasoning through that is an additional paradigm to the holistic evaluation you can have. s108 00:10:30.240 --> 00:10:40.839 And the idea here is do a coding evaluation, do a writing evaluation, do humanities last exam, and try to have as many paradigms as possible, and that's what I'm trying to build with E-Slams, right? s109 00:10:40.839 --> 00:10:45.560 So, we support about 50 games and 90 different like providers. s110 00:10:45.560 --> 00:10:48.880 We have chess and Go, which looks at planning invisible state. s111 00:10:48.880 --> 00:10:51.800 We have poker, which is decision-based on hidden information. s112 00:10:51.800 --> 00:10:54.600 And we have cooperation games based on shared goals and cooperation. s113 00:10:54.600 --> 00:10:58.000 And we also have trials for arcade games, right? s114 00:10:58.000 --> 00:11:01.200 And all of this tells you a lot about models and how they think. s115 00:11:01.200 --> 00:11:07.160 And on our platform, we have preview on eslams.com of how you can use the repo. s116 00:11:07.160 --> 00:11:12.760 And this is an example of Fable 5.1 playing tic-tac-toe against GPT-6 Astra, right? s117 00:11:12.760 --> 00:11:18.520 And you can look at how Fable and Astra are looking at each other's turn, and how they go ahead and play this. s118 00:11:18.520 --> 00:11:20.760 And this match, interestingly, ends in a tie. s119 00:11:20.760 --> 00:11:27.440 And most cases, a model wins, but Fable and Astra ended up knocking their heads to each other, right? s120 00:11:27.440 --> 00:11:30.000 And let's say you are deploying an agent in production. s121 00:11:30.000 --> 00:11:34.920 You can You want to go ahead and play against these AI models and see how they are. s122 00:11:34.920 --> 00:11:40.240 You can come on Islam.com and like play against these AI models on any different game that you want to. s123 00:11:40.240 --> 00:11:47.360 And you can all do all of this using the open source repo and make it play with your own agents and like evaluate your systems, right? s124 00:11:47.360 --> 00:11:59.120 And we are going to host the Grand Slams of AI on in February and the idea is to put an Olympics of for AI models to go ahead and compete and know who will stand out of all of this, right? s125 00:11:59.120 --> 00:12:07.120 So, please do check out Islam to know where we're going with it and this could be one paradigm of model evaluation you can add into your stack. s126 00:12:07.120 --> 00:12:11.040 So, now coming back, the main key part after all of this, right? s127 00:12:11.040 --> 00:12:17.960 Like you keep evolving, you keep adding paradigms, is to make sure that you keep testing the evaluation workflow, right? s128 00:12:17.960 --> 00:12:21.920 And the most important part of that is keeping what works in the flow. s129 00:12:21.920 --> 00:12:26.200 And my best example of this is Google Assistant moving to Gemini. s130 00:12:26.200 --> 00:12:29.200 I used to use Google Assistant for all of my reminders. s131 00:12:29.200 --> 00:12:36.200 And one day it moved to Gemini and it stopped giving me reminders and it said it's a language model that cannot set up reminders, right? s132 00:12:36.200 --> 00:12:43.839 So, it was excellent in the old Google Assistant model they had and before they moved it to Gemini and it stopped working and this is an example of Google s133 00:12:43.839 --> 00:12:46.280 not doing eval on what used to work, right? s134 00:12:46.280 --> 00:12:48.240 When they went ahead and launched the Gemini product. s135 00:12:48.240 --> 00:12:53.800 And that's extremely important to go ahead and make sure that you keep testing like the regression cases, right? s136 00:12:53.800 --> 00:13:04.880 And known task and past failures because a model that is extremely better than the old one could very well be incapable of something the old one did very well, for example, setting a reminder, s137 00:13:04.880 --> 00:13:05.640 right? s138 00:13:05.640 --> 00:13:13.280 And um fresh challenge cases at all points goes back to what I was saying in terms of setting the most ambitious target. s139 00:13:13.280 --> 00:13:16.480 The idea is that you have to keep pushing your model. s140 00:13:16.480 --> 00:13:25.480 If you feel that your model your is getting to an 85 or a 90 on your eval test, it's time that you better your eval and like push the frontier, right? s141 00:13:25.480 --> 00:13:32.840 And make sure that your model is challenged and has cases where it is not able to perform to expectations. s142 00:13:32.840 --> 00:13:35.080 And at all points, right? s143 00:13:35.080 --> 00:13:42.800 And I think this has been true in software throughout history is that you have to turn real-world failures into future tests, right? s144 00:13:42.800 --> 00:13:48.880 Your customer and your user is the most important evaluator of your agent and your model. s145 00:13:48.880 --> 00:13:57.280 So, every failure should have a proper workflow for it to enter into your system and make sure that like be that, right? s146 00:13:57.280 --> 00:14:02.160 So, one is to keep observing and there are multiple ways of making sure that you can observe. s147 00:14:02.160 --> 00:14:16.120 There are multiple observability tools that give you automatic data, give option for your users to report it to you with your simple thumbs up or a thumbs down, and collect as much data where your customers are telling you where there is a failure in the s148 00:14:16.120 --> 00:14:17.480 pipeline, right? s149 00:14:17.480 --> 00:14:24.120 Classic QA on software, go ahead and reproduce that issue, and try to understand where is the model failing? s150 00:14:24.120 --> 00:14:25.040 Is it a tool call? s151 00:14:25.040 --> 00:14:30.760 Is it Is it And what part of your eval suit failed in going ahead and doing that, right? s152 00:14:30.760 --> 00:14:35.040 And review it, add it to the pipeline, and add that to the eval pipeline. s153 00:14:35.040 --> 00:14:45.080 This becomes extremely important to keep this feedback loop going so that you go ahead and access new frontiers of things that you are missing out in every point in time. s154 00:14:45.080 --> 00:15:00.280 And I think the two ways to go ahead and get to eval that is as holistic as possible is one to consistently human review a process, and then ensure that your customer feedback is part of your eval pipeline, s155 00:15:00.280 --> 00:15:00.440 right? s156 00:15:00.440 --> 00:15:08.880 Because these are the last 5% edge cases that really come out which a model or an existing eval framework might not catch. s157 00:15:08.880 --> 00:15:12.600 So, to conclude, test the job and not just the answer, right? s158 00:15:12.600 --> 00:15:15.320 And that's extremely key to the whole process. s159 00:15:15.320 --> 00:15:22.920 And I'll just wrap up with all the things that we've discussed is one is always define your most ambitious success. s160 00:15:22.920 --> 00:15:25.480 Think about what life would be in AGI, right? s161 00:15:25.480 --> 00:15:39.560 And think about life in a model that has no inabilities and can absolutely perform like the human that you're trying to sort of emulate and try to define in this process, right? s162 00:15:39.560 --> 00:15:44.560 Second is continuously test the workflow at every part of the workflow. s163 00:15:44.560 --> 00:15:51.320 Don't assume things in the workflow or like assume for an input and output to be the outcome of anything, right? s164 00:15:51.320 --> 00:16:01.320 And when you test the workflow, I think something I slightly missed on here is to make sure that your your tests are designed for what that element of the workflow does. s165 00:16:01.320 --> 00:16:14.920 For example, let's say in that in your workflow there's a specific part that is doing tool calls consistently to get the information, your eval has to be designed for tool calling and not for any different process s166 00:16:14.920 --> 00:16:15.920 separately, right? s167 00:16:15.920 --> 00:16:22.800 Because the objective in that workflow process is to know what is your tool call ability and the tool tool call accuracy, right? s168 00:16:22.800 --> 00:16:24.080 At that at that point. s169 00:16:24.080 --> 00:16:35.080 So, every time make sure that when you're testing the workflow, you design the test to that specific part of the workflow and measure what is the success in that element of the workflow as well. s170 00:16:35.080 --> 00:16:38.960 And continuously gather evidence holistically. s171 00:16:38.960 --> 00:16:45.040 In an agentic world, what evidence for agentic action eval is something that keeps evolving. s172 00:16:45.040 --> 00:16:53.480 Uh and games is something I'm exploring to see how it works out, but there are so many different frontiers for you to go ahead and look at, right? s173 00:16:53.480 --> 00:16:58.800 Something super interesting that I came across while here I'm here is the parallel search model index, right? s174 00:16:58.800 --> 00:17:03.960 And how models are good at fetching search from parallels like AI search index. s175 00:17:03.960 --> 00:17:09.680 And that becomes very important for you to evaluate when you are designing a model that needs real-time data, right? s176 00:17:09.680 --> 00:17:20.959 So, make sure that you as a company or a person building these products is consistently looking for more evidence and more arenas and more paradigms for you to evaluate your models s177 00:17:20.959 --> 00:17:26.480 and that is one sure shot way to making sure that you cover most edge cases. s178 00:17:26.480 --> 00:17:42.240 The fourth one is to evolve the test as time flow and making sure that you cover all aspects from the past and the future into the test and the simplest thing is if your test is scoring high is it 85 90 that is the time you change your test and you have to make it harder. s179 00:17:42.240 --> 00:17:42.680 Right? s180 00:17:42.680 --> 00:17:52.400 The simplest thing is if you think everyone is scoring like in the 99th percentile for the SATs, it is time that the test have to change in the internet of itself, right? s181 00:17:52.400 --> 00:17:54.880 And always learn from reality. s182 00:17:54.880 --> 00:18:06.680 Reality is going to be your best teacher and the idea of eval that I started off with is to make sure that we deliver the promise that we are making to the consumer and the user in production s183 00:18:06.680 --> 00:18:13.000 and that is most defined in the customer feedback and the reality that really happens, right? s184 00:18:13.000 --> 00:18:24.360 So, consistently think about how you can incorporate real user feedback and real world test and how you can have human review and human testing into the workflow as you go ahead and get started. s185 00:18:24.360 --> 00:18:33.280 I think with these five thought processes and the idea that you go ahead and test the job and not just the answer, you will have a good mental framework s186 00:18:33.280 --> 00:18:35.960 to go ahead and build eval for high stake applications. s187 00:18:35.960 --> 00:18:36.800 Thank you for coming. s188 00:18:36.800 --> 00:18:40.280 I'm happy to catch up for questions afterwards.