WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=2aS7aKoXn64 NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/2aS7aKoXn64.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:12.480 --> 00:00:14.240 It's uh great to see all of you here today. s2 00:00:14.240 --> 00:00:18.320 We're super excited to talk about one of our favorite topics uh here at Theta. s3 00:00:18.320 --> 00:00:21.119 Um before we get started, we just want to introduce ourselves. s4 00:00:21.119 --> 00:00:25.359 Um so, hi, I'm a co-founder and CTO at Data Software. s5 00:00:25.359 --> 00:00:26.160 Hi, I'm Ryan. s6 00:00:26.160 --> 00:00:28.080 I'm a co-founder and CEO at Thata Software. s7 00:00:28.080 --> 00:00:34.399 Prior to this, I was previously a founding engineer at Deep Silken where we did research into turnary models. s8 00:00:34.399 --> 00:00:35.200 Awesome. s9 00:00:35.200 --> 00:00:37.280 So, I can get us started with the topic today. s10 00:00:37.280 --> 00:00:41.920 Um, we're going to be talking about oral environments uh within the context of long horizon tasks. s11 00:00:41.920 --> 00:00:48.239 And I think the most important thing for us to start with at the beginning is just talk about the trends and what long horizon actually means. s12 00:00:48.239 --> 00:00:54.960 So, you know, we all know that the horizon at which AI agents can work autonomously is accelerating really fast. s13 00:00:54.960 --> 00:00:59.039 Um this is just some of the metrics that you can look at to see how this progress is really accelerating. s14 00:00:59.039 --> 00:01:03.520 Um but I think it's really important to actually define what the time horizon here actually means. s15 00:01:03.520 --> 00:01:08.560 Uh we've gotten this data from one of the most common benchmarks out there that you've probably heard of for time horizons. s16 00:01:08.560 --> 00:01:10.479 I'm sure you've seen in your Twitter feed all the time. s17 00:01:10.479 --> 00:01:13.520 It's um comes from meter. s18 00:01:13.520 --> 00:01:20.799 Uh and meter kind of has one response or answer to this really important question about how do we actually define long horizon? s19 00:01:20.799 --> 00:01:27.360 Um, it's really important to understand because what we consider long horizon a year ago probably isn't really long horizon in our definition today. s20 00:01:27.360 --> 00:01:30.320 And what's long horizon today probably won't be long horizon in a year or two. s21 00:01:30.320 --> 00:01:35.439 Um, and I think that gets to our first point which is that long horizon is really kind of a scalar metric. s22 00:01:35.439 --> 00:01:46.000 Uh, it's useful for kind of measuring relative tasks like one task might be more long than another, but it's really hard to define into kind of a binary category of this task is long and this task is not. s23 00:01:46.000 --> 00:01:49.360 Uh, especially as the kind of scope changes over time. s24 00:01:49.360 --> 00:02:04.960 So I think the first way we can talk about defining this is how meter kind of looks at it which is human horizon meaning can we use humans as a benchmark of oh this task takes humans a certain amount of time so if AI agents can do that then they've reached this certain critical s25 00:02:04.960 --> 00:02:10.959 uh level of kind of a time horizon uh and the way meter kind of does it is they have thresholds for the tasks they care about. s26 00:02:10.959 --> 00:02:21.760 So, you know, they have a 50% threshold, meaning, you know, if a certain model reaches a 16- hour threshold on this benchmark, that means it can achieve tasks with a 50% success rate that take a human 16 hours. s27 00:02:21.760 --> 00:02:28.480 And, you know, there's a really rigorous methodology of how they actually measure how did it take human 16 hours, but we will kind of avoid some of those details. s28 00:02:28.480 --> 00:02:36.800 Um, the other way we usually think about what long horizon actually means is not with the reference of humans, but instead the reference of models. s29 00:02:36.800 --> 00:02:40.720 Uh so some of the relevant model units we usually care about are things like tokens. s30 00:02:40.720 --> 00:02:45.519 How many tokens are consumed in a trajectory, how many steps it took, how many tool calls it kind of takes. s31 00:02:45.519 --> 00:02:47.760 Um and these can be really noisy, right? s32 00:02:47.760 --> 00:02:49.599 Because I'm sure you guys have used different models. s33 00:02:49.599 --> 00:02:54.560 Um like you know a lot of the codeex models are seen as more token efficient than some of the cloud models. s34 00:02:54.560 --> 00:02:57.519 Um and it's a pretty noisy estimate for a couple reasons. s35 00:02:57.519 --> 00:03:04.560 One is that which model you're using like I just said and different harnesses you care about have a pretty big impact on how many tokens are actually consumed on a task. s36 00:03:04.560 --> 00:03:04.879 Right? s37 00:03:04.879 --> 00:03:09.440 So, um, this can be pretty hard to interpret when you're not holding variables constant. s38 00:03:09.440 --> 00:03:18.640 Um, you know, if a task takes GPT model 500,000 tokens, that doesn't really tell you a lot about what that task would look like for cloud models until you actually run on those cloud models. s39 00:03:18.640 --> 00:03:33.360 But despite it being a pretty noisy metric, it's actually really useful uh and important for us to understand because, you know, the amount of tokens that are consumed tells us a lot about how difficult a task actually is for an AI agent to kind of tackle it autonomously, right? s40 00:03:33.360 --> 00:03:36.400 uh we have to deal with things like compaction over long horizons. s41 00:03:36.400 --> 00:03:41.760 Um you know they don't really stay coherent over enough steps or trajectory uh length that you kind of achieve. s42 00:03:41.760 --> 00:03:55.120 So even though it's kind of a noisy metric uh it can be really useful when you know if we look at what a GPT 5.5 model can do uh now and then you use the same model generation and kind of see oh now can actually achieve a a million trajectory based on an increased context window or improved s43 00:03:55.120 --> 00:03:56.000 compaction endpoint. s44 00:03:56.000 --> 00:04:02.159 That tells us a lot about how autonomous AI agents can actually go for long periods of time in that sense. s45 00:04:02.159 --> 00:04:07.439 Um, and it really defines for us what the technical frontier actually means uh for models right now. s46 00:04:07.439 --> 00:04:09.120 Maybe not really human adjacent. s47 00:04:09.120 --> 00:04:15.360 It's really hard to say how many tokens a task takes for a human because we don't really think in tokens, but still very useful in that kind of sense. s48 00:04:15.360 --> 00:04:18.239 Um, so these are two different approaches we can think about. s49 00:04:18.239 --> 00:04:20.880 But what's actually the right way to think about this? s50 00:04:20.880 --> 00:04:23.600 Uh, the answer is that we probably want to think about all of these. s51 00:04:23.600 --> 00:04:28.240 And if we just look at one of these metrics in isolation, it's probably not a great way of measuring things. s52 00:04:28.240 --> 00:04:38.560 So, I went through some of the weaknesses with measuring with like model specific metrics like tokens and and steps, but there's also a lot of weaknesses in the other approach of kind of relying on humans. s53 00:04:38.560 --> 00:04:45.759 Um, you know, what's long horizon for a human isn't necessarily that difficult for a model depending on what the actual task you care about is. s54 00:04:45.759 --> 00:04:48.560 You know, there's a lot of tasks that are really tedious and time inensive. s55 00:04:48.560 --> 00:04:55.600 Maybe like, you know, some financial analyst has to go into like an Excel file and fix a bunch of formatting issues throughout uh the the task. s56 00:04:55.600 --> 00:05:00.400 Maybe they're changing like the theming of like the colors in the in the actual file, right? s57 00:05:00.400 --> 00:05:02.240 That might be really tedious for a human to take. s58 00:05:02.240 --> 00:05:09.840 It might take them like days to do that if it's a really big Excel file, but for a model, it can maybe write a Python script or find some other cool trick to do that really quickly. s59 00:05:09.840 --> 00:05:16.639 And that's not really hard for it to do, but you never really expect a financial expert to do that because, you know, they most of them don't really know how to write these Python scripts. s60 00:05:16.639 --> 00:05:19.520 Um, so I think that's one thing to note. s61 00:05:19.520 --> 00:05:25.280 And the other that I kind of briefly touched upon before is that the methodology of how we actually measure this has a really big impact. s62 00:05:25.280 --> 00:05:31.600 And if you, you know, someone is out there saying, "Hey, we have some tasks or environments that are 16 hours long on average, someone else is 20 hours." s63 00:05:31.600 --> 00:05:37.759 That's really hard to compare across people because there's so many different things in the methodology that really impact uh kind of what that actually means. s64 00:05:37.759 --> 00:05:40.400 It can mean, you know, the quality of the experts you're using. s65 00:05:40.400 --> 00:05:46.000 Some more experienced experts might actually be way more efficient at doing a certain type of financial or coding task, whatever it kind of is. s66 00:05:46.000 --> 00:05:53.520 Um and I think this becomes really really important as we start shifting towards uh kind of the frontier of even human capabilities. s67 00:05:53.520 --> 00:06:06.960 So you know as this meter talks about this but as you shift towards more long resin tasks and tasks that only the top 10% the top 1% top.1% of humans can really do these estimates start to get really really noisy and it's something that we really have to consider. s68 00:06:06.960 --> 00:06:16.160 Uh and I think you know the way agents work is kind of developing in its own separate path and there are a lot of different bottlenecks and different things that AI agents are better at than even the way humans work. s69 00:06:16.160 --> 00:06:23.520 And with that in mind uh you know as these paths kind of diverge of how humans do work and what their limitations are and what agents do and what their limitations are. s70 00:06:23.520 --> 00:06:34.639 Uh it's really important to kind of keep both these metrics in mind because they kind of tell and paint different pictures uh of of what's actually relevant and and you don't really get the whole picture by just looking at one in that sense. s71 00:06:36.319 --> 00:06:36.720 Yep. s72 00:06:36.720 --> 00:06:40.639 So now the question becomes, how do you measure model capabilities? s73 00:06:40.639 --> 00:06:46.560 And this is a really important question because fundamentally long horizon tasks aren't the only thing we care about. s74 00:06:46.560 --> 00:06:53.120 This is the larger question that we want to think about every time we're trying to create tasks, create environments to train our models. s75 00:06:53.120 --> 00:06:57.600 And so the first way we can think about this is environment complexity. s76 00:06:57.600 --> 00:07:01.680 And specifically environment complexity related to tool coordination, right? s77 00:07:01.680 --> 00:07:05.680 So how many tools or external dependency does the agent have to coordinate? s78 00:07:05.680 --> 00:07:09.759 How many tools or external dependencies does the agent have to move information across? s79 00:07:09.759 --> 00:07:24.080 So if we start off kind of thinking about what the world looked like before a long horizon task uh you know world, we'll notice that there was you know a low complexity world where the agent maybe had to read one file or one set of files in a code base and that's kind of what a task entailed. s80 00:07:24.080 --> 00:07:35.840 But now we can see increasingly as these tasks become more long horizon what is important to define for measuring model capabilities is okay the the agent should be using a ton of different tools like graphana for observability s81 00:07:35.840 --> 00:07:48.080 to parse logs or GitHub for CI/CD or AWS cloudatch or reading and writing to a database and we're going to notice that as we sort of start to have these agents and these environments s82 00:07:48.080 --> 00:07:57.280 use many tools that we also start to think about environment complexity in regards to state changes which is effectively the degree to which the environment changes throughout the task. s83 00:07:57.280 --> 00:08:03.919 And so fundamentally the way we want to think about this is right all long horizon tasks aren't equal. s84 00:08:03.919 --> 00:08:11.919 So for example, one task can you know maybe be made by artificially long horizon by chaining together unrelated independent tasks. s85 00:08:11.919 --> 00:08:17.680 However, that doesn't actually tell us or meaningfully measure the model capabilities. s86 00:08:17.680 --> 00:08:25.360 Instead, a key component of this is actually being able to have the earlier decisions in the in the environment influence the later decisions. s87 00:08:25.360 --> 00:08:34.080 And this comes back to how the agents are asked to interact with the tools, how these tools change the state of the environment, etc. So, we can look at an a concrete example for this. s88 00:08:34.080 --> 00:08:46.399 One example where you'll see paralyzable complexity, which is effectively not involving a lot of state changes, is when you can maybe have an agent analyzing a large code base and then the agent needs to spawn off multiple sub aents and it can very easily paralyze this, right? s89 00:08:46.399 --> 00:08:52.320 it can look at a lot of the different files in parallel, come back to the to to the master agent and then kind of wrap this all up, right? s90 00:08:52.320 --> 00:09:05.360 But meanwhile, if we look at sequential complexity, we'll see if you have to use a dashboard or logs, a bad early query or a misread can cascade into these downstream steps that really start to have major consequences later on, right? s91 00:09:05.360 --> 00:09:09.839 It's all dependent on how you use those tools and how the state of the environment changed. s92 00:09:09.839 --> 00:09:15.360 So the third area that we also need to consider for measuring model capabilities is ambiguity, right? s93 00:09:15.360 --> 00:09:20.959 And ambiguity is defined as the information you give the agent and the environment when starting the task. s94 00:09:20.959 --> 00:09:28.160 So this could be the instructions, this could be the artifacts, etc. And increasingly as these agents work with more artifacts at the start, right? s95 00:09:28.160 --> 00:09:30.959 We want to have them mirror the work that humans really do. s96 00:09:30.959 --> 00:09:35.760 And the work that humans really do has a lot to deal with ambiguity, right? s97 00:09:35.760 --> 00:09:40.959 They always are are don't have the most complete information and they want to let exploration happen. s98 00:09:40.959 --> 00:09:50.720 And so we believe that to measure model capabilities, we need to test the model's ability to explore and explore throughout the environment as well and explore these artifacts similar to how a human would. s99 00:09:50.720 --> 00:09:59.040 Now the trade-off with this, right, is that if you are going to have ambiguity in the materials you give, there's a lot more possible paths that the agent could take. s100 00:09:59.040 --> 00:10:01.040 There's a lot more ways the agent could be right. s101 00:10:01.040 --> 00:10:05.920 And that means that standardized evaluation gets much, much harder. s102 00:10:06.080 --> 00:10:06.880 Awesome. s103 00:10:06.880 --> 00:10:16.000 So I'm going to talk about one of the hardest things there are to build environments and one of the most complex things to really think about where there's a lot of nuance which is the verifier in the environment. s104 00:10:16.000 --> 00:10:20.880 How do we actually know that the work the agent did was correct and give it some reward signal during the training process. s105 00:10:20.880 --> 00:10:22.880 So I think there's a few challenges here. s106 00:10:22.880 --> 00:10:30.480 Um you know I think just to give a high level overview um you know tasks are getting more complex the environments are getting more complex the trajectories are getting longer s107 00:10:30.480 --> 00:10:45.040 and we've shifted a lot from you know a lot of the early RL that we were doing in in recent times was really in hard verifiable domains and that's why we saw these gains in in math and kind of uh like data structure style coding problems but what's happened over time is now we really care about a s108 00:10:45.040 --> 00:10:58.640 bunch of economically valuable work in software file domains is is a way to put it where uh you know we can't just run a Python script or run test cases is or or write a proof to really see whether or not the output was correct or whether or not the environment was changed correctly. s109 00:10:58.640 --> 00:11:10.079 We have to start using other techniques and the main way we're really going to use that is kind of introduce a judge model or critic model as some people put it and they kind of can add a lot of nuance to how we actually look at a few things here. s110 00:11:10.079 --> 00:11:14.399 Um you know very critical for how we actually determine correctness and assign reward. s111 00:11:14.399 --> 00:11:15.920 Um they'll look at two things mainly. s112 00:11:15.920 --> 00:11:20.320 One is usually either the state final state of the environment and kind of how it was impacted. s113 00:11:20.320 --> 00:11:29.920 Um and the other is looking at the trajectory of how the model that you're actually training made kind of changes to the the state of the environment as well and and what kind of correctness look like there. s114 00:11:29.920 --> 00:11:37.279 Um so you know why do we actually use uh judges and usually rubrics um as a technique? s115 00:11:37.279 --> 00:11:41.440 I think it's really important to understand before we can even understand how to use them properly which you know I think there's a few reasons. s116 00:11:41.440 --> 00:11:50.079 One is that like I said for these software fellow domains there's like an entire class of problems that are really important and a lot of the problems we care about that really you can't really write a deter deterministic s117 00:11:50.079 --> 00:11:56.560 verifier for um they would be really impractical brittle or just downright impossible depending on what the problem setup really is. s118 00:11:56.560 --> 00:12:07.760 Um and you know I think the other thing also is that like I said we're going to look at the trajectory uh and you know not all solutions are really created equal and not all paths of those solutions are equal either. s119 00:12:07.760 --> 00:12:11.680 uh you know the worst case of a bad solution we can get is some reward hacking that happens. s120 00:12:11.680 --> 00:12:15.760 Lots of different types of reward hacking can happen depending on the setup or the task you kind of care about. s121 00:12:15.760 --> 00:12:22.720 You know agent can escape a sandbox maybe see privilege information it shouldn't be seeing about maybe a hidden test suite for like a coding task. s122 00:12:22.720 --> 00:12:28.399 Um this is all behavior that we want to prevent obviously because those are not actually really valid solutions we really care about. s123 00:12:28.399 --> 00:12:36.959 Um, and you know, a lot of this mitigating this is going to require strengthening your verifier and your environment setup, but the judge is really, really important in actually catching this behavior. s124 00:12:36.959 --> 00:12:41.760 And that's an important reason of why we actually look at the trajectory that the agent actually took to get there. s125 00:12:41.760 --> 00:12:45.839 Um, yeah, and I think there's a lot of careful things you want to be doing here. s126 00:12:45.839 --> 00:12:53.279 Uh, one is that there's nuance in how much guidance or uh like explicit rigidness you want to add to the trajectories that the model can actually take. s127 00:12:53.279 --> 00:13:00.160 um if we kind of enforce this too tightly, we collapse the state space of how many actual paths the agent actually explores. s128 00:13:00.160 --> 00:13:12.240 Uh and that can be really bad, especially because I think some of the more simple approaches we've seen with judges early on is, hey, we'll just give it a reference answer or solution or maybe a sample trajectory of what a good solution looks like and then just compare against s129 00:13:12.240 --> 00:13:14.560 uh what the model did and say, hey, does it match up with that? s130 00:13:14.560 --> 00:13:19.680 Uh and that really does not work for these more ambiguous or open-ended tasks because there's so many possible correct solutions. s131 00:13:19.680 --> 00:13:22.320 It's basically impossible to account for every single one. s132 00:13:22.320 --> 00:13:26.800 and we want to check for more robust methods that allow for these different solutions. s133 00:13:28.079 --> 00:13:37.279 So now that we've kind of established why we use judges, uh we want to go through some of the general heristics and kind of principles we think about when we're designing good judges. s134 00:13:37.279 --> 00:13:39.440 Um some of the things that we think about at data. s135 00:13:39.440 --> 00:13:45.839 So you know I think the first important uh consideration to make is that judges are agents too. s136 00:13:45.839 --> 00:13:55.680 Um, you know, so as environments get really complex, oftent times we a consideration we kind of have is like, hey, we have to make sure the harness can kind of scale and and kind of match up with whatever environment s137 00:13:55.680 --> 00:13:56.480 uh you kind of have. s138 00:13:56.480 --> 00:14:00.160 Maybe that means introducing a bunch of new tools and making sure your harness can support those tools really well. s139 00:14:00.160 --> 00:14:03.120 The agent has clear observability over what's happening in the environment. s140 00:14:03.120 --> 00:14:10.320 Um, but I think like we said, the way the judge determines correctness is that it oftent times has to look at the state of the environment itself as well. s141 00:14:10.320 --> 00:14:15.040 So a lot of the harness that you've designed for the agent might also be reused uh for the judge as well. s142 00:14:15.040 --> 00:14:17.760 Um, I think the best way to illustrate this is the example we have here. s143 00:14:17.760 --> 00:14:27.360 Let's say you've defined a task where, you know, there's some deployment failure with the software engineering task of some platform you're deploying and the agent's task is to like sift through the CI/CD logs on GitHub, s144 00:14:27.360 --> 00:14:37.519 look through the cloud cloudatch logs, figure out whatever happened, uh, kind of apply the changes you care about to the codebase and then open a PR and and kind of kick off a redeploy there once the PR is merged. s145 00:14:37.519 --> 00:14:50.160 Um for a lot of for a lot of that if the judge actually wants to verify uh whether or not this is correct besides just like looking at the tool calls agent mate which are usually not very reliable it actually has to also check the GitHub logs it might check the AWS s146 00:14:50.160 --> 00:14:54.720 logs or the GitHub logs after the deployment happened to make sure oh are things actually working properly. s147 00:14:54.720 --> 00:15:00.240 So it's really important that the judge has access to the environment in the same way uh with some important safeguards of course. s148 00:15:00.240 --> 00:15:04.959 One is that we don't want the judge to make an accidental mutation in some way to the environment after the agent is done. s149 00:15:04.959 --> 00:15:06.320 So you want to be very careful about that. s150 00:15:06.320 --> 00:15:09.040 Maybe that means enforcing readon permissions for a lot of this information. s151 00:15:09.040 --> 00:15:11.040 It can't actually kick off a deployment or anything like that. s152 00:15:11.040 --> 00:15:12.880 So those are things to be careful about. s153 00:15:12.880 --> 00:15:20.320 But I think this is really really important especially where there's a lot of open-ended approaches and the only way we can really verify correctness is to actually look at the state itself. s154 00:15:20.320 --> 00:15:24.320 Uh you the answer isn't obvious of whether or not the agent completed the task just from looking at the trajectory. s155 00:15:24.320 --> 00:15:27.519 So I think that's one example where this approach is really really important. s156 00:15:27.519 --> 00:15:42.560 Um I think uh the other thing to be notable of is you know as these environments get more complex the agent trajectories get longer and longer and part of the reason we also need the judge to be an agent is that you can't just use this really basic approach of taking the trajectory and stuffing it in the s157 00:15:42.560 --> 00:15:45.519 context window of the judge and kind of have it be a basic LM call. s158 00:15:45.519 --> 00:15:49.040 Uh these trajectories can get really really long and really really complex. s159 00:15:49.040 --> 00:15:54.720 So we need to do a lot more thoughtful uh processing of the trajectory in some meaningful way. s160 00:15:54.720 --> 00:15:57.600 So you know that might mean we put into some database. s161 00:15:57.600 --> 00:16:00.160 We use sub agents to actually enrich certain information. s162 00:16:00.160 --> 00:16:03.600 Maybe we parse out specific phases that the agent was actually in. s163 00:16:03.600 --> 00:16:05.680 Maybe the beginning part was it going through logs. s164 00:16:05.680 --> 00:16:07.360 The second part was actually writing code. s165 00:16:07.360 --> 00:16:09.759 The third part was actually it checking what happened after that. s166 00:16:09.759 --> 00:16:11.600 These are all different things we want we want to do. s167 00:16:11.600 --> 00:16:14.480 And in that sense we need to make the trajectory itself queryable. s168 00:16:14.480 --> 00:16:19.440 So that might mean enriching of information like I just said or some other metadata we can kind of look at at certain steps. s169 00:16:19.440 --> 00:16:26.560 is really important so the agent can find critical steps um you know like failure points and and verify whether or not those are actually failures. s170 00:16:26.560 --> 00:16:31.519 Making that uh kind of usable for the agent is really really important. s171 00:16:32.720 --> 00:16:36.560 Um I think another important thing to consider is learnability of our environments. s172 00:16:36.560 --> 00:16:43.519 Um you know the most important thing here is just the density of the reward signal and a lot of that comes from your rubric and kind of how the judge is defining that. s173 00:16:43.519 --> 00:16:54.880 So, I think if you be very careful with just uh you know overloading with density in in your rubric, a lot of times, especially for frontier problems that models aren't really capable of yet, judges will really struggle to apply that rubric consistently. s174 00:16:54.880 --> 00:16:59.120 So, there's a lot of QA we kind of need to do to make sure judges are able to apply that information correctly. s175 00:16:59.120 --> 00:17:06.400 Um, you know, there's other learnability factors that we care about and we measure in environments like the distribution of tasks and and and the actual underlying data there is. s176 00:17:06.400 --> 00:17:15.760 Uh and and these are all kind of things we think about for learnability and it's really important otherwise you're just wasting a bunch of compute on on problems where the model can't actually effectively learn. s177 00:17:16.880 --> 00:17:19.839 Um you know these are some emerging rubric judge patterns we've seen. s178 00:17:19.839 --> 00:17:21.039 I'll quickly skim over this. s179 00:17:21.039 --> 00:17:24.959 Um you know oftent times you deterministic verifiers aren't completely dead. s180 00:17:24.959 --> 00:17:26.959 oftentimes we use them in tandem with judges. s181 00:17:26.959 --> 00:17:36.720 Maybe generating an artifact for the judge to actually look over where you're maybe collecting metrics or interesting thing that we could also use is dynamic evaluation time rubrics where we're actually generating s182 00:17:36.720 --> 00:17:42.000 um you know we're maybe giving partial credit where we've baked in some assumptions that the models made and assume they're correct. s183 00:17:42.000 --> 00:17:47.200 It's like grading a test assuming like if you got the first part wrong, let's just assume it's correct, did they get the rest of the part right? s184 00:17:47.200 --> 00:17:51.919 That can be really important as well as well for kind of assigning credit there. s185 00:17:53.679 --> 00:17:55.120 skip over this part. s186 00:17:55.120 --> 00:17:58.960 Um I will let R just close things off with some things about QA for rubrics. s187 00:17:58.960 --> 00:17:59.440 Yep. s188 00:17:59.440 --> 00:18:02.640 So for each rubric we produce, we run a couple different tests. s189 00:18:02.640 --> 00:18:03.679 We won't go into all of them. s190 00:18:03.679 --> 00:18:04.960 Some of them are pretty basic, right? s191 00:18:04.960 --> 00:18:06.080 Gold, no op variance. s192 00:18:06.080 --> 00:18:18.320 These are tests you want to be considering regardless for your verifiers, but I think increasingly, you know, as as you involve AI in the process of even creating rubrics or verifying rubrics or aiding experts, you need to have more and more tests, especially as the tasks become more long horizon. s193 00:18:18.320 --> 00:18:21.520 And so that really touches on the coverage and the expert agreement. s194 00:18:21.520 --> 00:18:25.360 But I think what we wanted to close off with today is why a lot of this stuff matters, right? s195 00:18:25.360 --> 00:18:29.120 We spent a lot of time earlier in this presentation defining what long horizon means. s196 00:18:29.120 --> 00:18:39.600 And a huge reason we did that is because we feel like a lot of the literature and data, a lot of the literature shows that a lot of the data being produced right now and being used to train and evaluate models is actually flawed. s197 00:18:39.600 --> 00:18:44.080 So we present three major benchmarks in the area of finance predominantly. s198 00:18:44.080 --> 00:18:47.120 And so this is GDP valer toolbench and Apex agents. s199 00:18:47.120 --> 00:18:49.039 There's a couple of notable issues here. s200 00:18:49.039 --> 00:19:01.200 First, if you look at the average human hours per task, based on what Meter has defined for a lot of the leading frontier models, a lot of these different average human hours per task fall far below that and so they wouldn't actually be considered long horizon tasks. s201 00:19:01.200 --> 00:19:10.799 The second notable issue here, right, is that we see that these benchmarks are already reasonably saturated and we think this is a downstream effect of the average human hours per task. s202 00:19:10.799 --> 00:19:13.440 So, it's really important to look at the metrics that are being used here. s203 00:19:13.440 --> 00:19:24.640 If you look at, you know, the Apex agents IB section of this benchmark that they put out, pass at one effectively means that for like 57% of cases, the tasks are 100% solved. s204 00:19:24.640 --> 00:19:30.960 That is effectively telling us that like there's a large part of these tasks that models are solving similar to what we've seen already. s205 00:19:30.960 --> 00:19:34.559 But I think a third key important part here is the breath. s206 00:19:34.559 --> 00:19:41.440 For each of these different uh benchmarks, particularly GDP val, they have a very narrow set of Excel tasks that they consider for finance. s207 00:19:41.440 --> 00:19:44.160 And for Apex agents, they're largely focused on IB. s208 00:19:44.160 --> 00:19:53.039 What this means is that a lot of these more important areas for learnability like, you know, credit, debt, risk in the domain of finance don't really get covered. s209 00:19:53.039 --> 00:19:59.120 And then I think lastly, I'll I'll I'll note there the reward signal as Gver mentioned is really important. s210 00:19:59.120 --> 00:20:09.120 And in regards to the reward signal here, we'll we'll notice that there's like really, you know, if you look at if you look at what you need for a rubric, you need very granular, s211 00:20:09.120 --> 00:20:10.559 detailed reward signal. s212 00:20:10.559 --> 00:20:17.520 You need, you know, we we have 20 different subriteria and 10 different subriteria per criteria. s213 00:20:17.520 --> 00:20:26.640 So, I think there's a lot of room that's left when you read these benchmarks into how granular reward signal they're giving, which is really important for being able to go ahead and train your models. s214 00:20:26.640 --> 00:20:30.400 With that, I think I wanted to round off with a couple of stats about the data we produced. s215 00:20:30.400 --> 00:20:33.200 Here we, you know, you can look at some statistics for our finance data. s216 00:20:33.200 --> 00:20:38.400 We can see that the human time to complete one task on average is 15 hours over a 50 task sample set. s217 00:20:38.400 --> 00:20:41.679 Furthermore, it takes models a pretty long time to work through these tasks. s218 00:20:41.679 --> 00:20:47.200 And after all of that, across all the domains we care about within finance, for example, they still struggle significantly. s219 00:20:47.200 --> 00:20:52.720 And so here we provide mean five notably different than, you know, all of these previous scores uh we see here. s220 00:20:52.720 --> 00:20:56.640 So, thanks for for taking the time to talk with us today. s221 00:20:56.640 --> 00:20:58.880 Thank you.