WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=xo68uCibfm8 NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/xo68uCibfm8.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:12.639 --> 00:00:13.519 Yep. s2 00:00:13.519 --> 00:00:14.480 Awesome. s3 00:00:14.480 --> 00:00:15.759 Thanks everyone for being here. s4 00:00:15.759 --> 00:00:17.119 My name is Kevin Madura. s5 00:00:17.119 --> 00:00:18.960 I'm from a company called Alex Partners. s6 00:00:18.960 --> 00:00:20.560 We're we're a consulting firm. s7 00:00:20.560 --> 00:00:23.600 Um I'm here to talk to you today about RLMs. s8 00:00:23.600 --> 00:00:28.320 Just curious, show of hands, who here is familiar with RLMs? s9 00:00:28.320 --> 00:00:29.840 So we know how much time to spend on it. s10 00:00:29.840 --> 00:00:30.080 Okay. s11 00:00:30.080 --> 00:00:31.199 So not many. s12 00:00:31.199 --> 00:00:31.439 All right. s13 00:00:31.439 --> 00:00:32.880 Well, that's good. s14 00:00:32.880 --> 00:00:36.719 So we'll start with what an RLM is and and why it's different. s15 00:00:36.719 --> 00:00:39.280 So RLM is recursive language model. s16 00:00:39.280 --> 00:00:48.879 And really the the key difference here is that it treats the context as an object that it can interact with symbolically in its environment. s17 00:00:48.879 --> 00:00:59.760 So it differs from a tool call in the sense that typically when you do a tool call it's JSON or some type of string that's being sent being interpreted elsewhere s18 00:00:59.760 --> 00:01:03.920 maybe by some other program and that's that's returning effectively as a string. s19 00:01:03.920 --> 00:01:08.320 The key difference here is that it's interacting with a symbolic environment. s20 00:01:08.320 --> 00:01:11.119 So typically that's a ripple Python ripple. s21 00:01:11.119 --> 00:01:14.000 Um so that's key difference number one. s22 00:01:14.000 --> 00:01:20.159 Key difference number two is that it's it has the ability to delegate to another LM often to itself. s23 00:01:20.159 --> 00:01:31.920 You can specify whether it's the same model or a different model, but fundamentally because it lives in this environment, you can offload or or make a sub call to another LM with particular parameters s24 00:01:31.920 --> 00:01:34.159 that also lives in that ripple environment. s25 00:01:34.159 --> 00:01:50.079 And so you get this ability to recursively decompose problems and apply and have the LLM basically decide how to apply certain logic or certain interpretations or write its own code to solve those problems s26 00:01:50.079 --> 00:01:51.439 and then that recurses down. s27 00:01:51.439 --> 00:01:57.280 So the subLM can do the same sort of thing in terms of understanding and interpreting what it thinks it needs to do. s28 00:01:57.280 --> 00:01:59.040 And I added this last one here. s29 00:01:59.040 --> 00:02:03.119 It's largely bitter lesson pilled in my opinion, right? s30 00:02:03.119 --> 00:02:06.640 is and and shared by Alex and the rest of the the creators of it. s31 00:02:06.640 --> 00:02:17.440 But as models get better, you should be able to defer more and more to the model for it to kind of figure out on its own what it needs to do. s32 00:02:17.440 --> 00:02:22.720 So the I don't know if this is the actual kind of starting point for RLMs. s33 00:02:22.720 --> 00:02:26.319 This is one that that I consider to be one of the the first kind of inklings of it. s34 00:02:26.319 --> 00:02:31.760 This is a tweet from Omar who is Alex's adviser um for for RLMs. s35 00:02:31.760 --> 00:02:43.360 And this was a concept that he had come up with where it was basically an ability to use DSPI and some other techniques to take in arbitrary um arbitrary length inputs. s36 00:02:43.360 --> 00:02:52.879 And basically the use case here would be summarizing an arbitrarily long document and coming up with a table of contents and some summary of of that content. s37 00:02:52.879 --> 00:03:00.000 But at least to me, this is kind of the first inkling of okay, context windows might not be something you need to deliberately manage. s38 00:03:00.000 --> 00:03:10.000 Although there's of course benefits to doing so, uh there there could be ways to um to exceed the the the context windows using some of these clever techniques. s39 00:03:10.159 --> 00:03:20.720 Uh and so if you read the paper and some of the blog posts that are out there from Alex and Omar, I mean it it it has demonstrabably better performance on some of these long context tasks. s40 00:03:20.720 --> 00:03:30.640 So, ulong is one benchmark where um the the intent of the benchmark it's is to measure model performance on answering questions about excessively long context. s41 00:03:30.640 --> 00:03:38.879 This another one browse comp where it needs to iterate through a large a large body and corpus of text and answer particular questions about it. s42 00:03:38.879 --> 00:03:41.680 You can see the blue line at the top there is the RLM. s43 00:03:41.680 --> 00:03:45.360 It's very good performance as compared to some of these other models. s44 00:03:45.360 --> 00:03:55.040 And even on on the price curve, the the purple is actually just using tool calling um with GBT5 calling a BM25 tool. s45 00:03:55.040 --> 00:04:00.239 And that's actually even more expensive for worse performance than than an RLM. s46 00:04:00.239 --> 00:04:05.120 So it's worth reading into if you're interested in in some of the benchmarks and how RLMs perform. s47 00:04:05.120 --> 00:04:15.680 Um but fundamentally uh an RLM again takes in your input and you're kind of deferring to the model about how to decompose the process what code it needs to write s48 00:04:15.680 --> 00:04:19.120 and it is very tightly integrated with the ripple itself. s49 00:04:19.120 --> 00:04:22.079 So it it by itself defines what it needs to do. s50 00:04:22.079 --> 00:04:32.479 And so I kind of had this mental model in terms of and I'm very DSP pipel if if you couldn't tell by now basically a student of of Omar and the rest of the group there where s51 00:04:32.479 --> 00:04:41.120 you have this relatively deterministic shell of what you want to do like what what is your intent what is your actual task that you're trying to accomplish. s52 00:04:41.120 --> 00:04:50.800 You define that in terms of your inputs and your outputs and some type of guidance or prompt or what have you to the model to say this is generally what I want to achieve. s53 00:04:50.800 --> 00:04:52.160 go off and do it. s54 00:04:52.160 --> 00:04:54.320 Here's the things that you can expect as your input. s55 00:04:54.320 --> 00:04:55.520 Here's what I want out of it. s56 00:04:55.520 --> 00:04:57.120 Go figure out the rest. s57 00:04:57.120 --> 00:05:07.199 And so this applies for using something like DSPI, but I think it applies to RLMs as well because you don't have to worry as much now about how the actual s58 00:05:07.199 --> 00:05:10.000 implementation works in the middle. s59 00:05:10.000 --> 00:05:17.120 You can just have some guarantees about the inputs and the outputs and you can let the model figure out the rest of of that part of it. s60 00:05:17.120 --> 00:05:29.440 So, a lot of this comes down to if if you were at um I think it was code in November in uh in New York City, Dex had this great talk about just broader context engineering and he he coined something like the dumb zone s61 00:05:29.440 --> 00:05:31.680 which it's kind of grayed out at the bottom there. s62 00:05:31.680 --> 00:05:35.120 But the point is that we all know that there's context rot, right? s63 00:05:35.120 --> 00:05:39.919 Once you fill up the context window to a certain degree, performance starts to degrade. s64 00:05:39.919 --> 00:05:52.000 And so RLMs somewhat get around this problem because the context itself doesn't fill up as quickly because you're deferring a lot of the subtasks to the subm models and it's s65 00:05:52.000 --> 00:05:58.240 the full kind of context and the inputs aren't exposed to the context window itself. s66 00:05:58.240 --> 00:06:05.680 It lives as a variable in the ripple and so the main LM can choose how to um how to access that. s67 00:06:05.680 --> 00:06:08.800 It can offload some of these subtasks to to sublim. s68 00:06:08.800 --> 00:06:13.520 And really the only context that it gets back are the things that actually matter. s69 00:06:13.520 --> 00:06:18.319 So in terms of how it's meaningfully different, rag of course you kind of just stuff the context window. s70 00:06:18.319 --> 00:06:20.080 You want it to limit there. s71 00:06:20.080 --> 00:06:29.919 Agents are largely just bringing strings back and you don't have this tight coupling between the logic, the execution and the results. s72 00:06:29.919 --> 00:06:32.960 And so you still run into the same sort of problem there. s73 00:06:32.960 --> 00:06:35.759 Same thing with tool calling and and codec. s74 00:06:35.759 --> 00:06:48.560 And then RLMs as I mentioned you're act the the LLM is actually just interacting with the context the results as variables in the ripple so that it can do additional computation on versus s75 00:06:48.560 --> 00:06:52.000 it trying to attend to all these different tokens in this in its context window. s76 00:06:52.000 --> 00:06:59.120 It's it's a meaningfully different way of of the LLM interacting with the uh the actual content itself. s77 00:06:59.120 --> 00:07:03.599 And so people always say, okay, what's the difference between that and and encoding agents? s78 00:07:03.599 --> 00:07:13.440 Um, in my mind, the largest difference is that the way that tool calls are tool calls calls are done is passing strings back and forth. s79 00:07:13.440 --> 00:07:26.720 Um but you can see with the release recently of workflows that uh Anthropic is doing something fairly similar and they um at the CIS conference I think it was Tar or someone similar s80 00:07:26.720 --> 00:07:34.800 um mentioned the RLM paper as a key driver of workflows and how they're how they've implemented it. s81 00:07:34.800 --> 00:07:44.000 And you can see here the intermediate results for workflows live in script variables i.e. a variable in the context. s82 00:07:44.000 --> 00:07:50.479 So, it's driving some of these these breakthroughs and some of these techniques from the from the labs as well. s83 00:07:53.199 --> 00:08:02.879 I'll skip through this a bit just because I have about 10 minutes left, but generally speaking, when you want to use it, it's obviously for large or dense input context. s84 00:08:02.879 --> 00:08:06.240 An underexplored area is outputs as well. s85 00:08:06.240 --> 00:08:16.720 So if you have a some type of task where you need to generate hundreds of thousands of lines or whatever it might be, RLMs I think would be a good candidate for that as well. s86 00:08:16.720 --> 00:08:19.840 Obviously tasks that are imunable to some type of decomposition. s87 00:08:19.840 --> 00:08:27.280 So if you want to look through the entire I don't know the whole tax code as an example and try and find loopholes or something. s88 00:08:27.280 --> 00:08:30.080 You can't obviously put all of that into context at once. s89 00:08:30.080 --> 00:08:38.719 You could use an LLM to crunch through all of that and iteratively explore and use sub agents to explore interesting areas of this of the tax code. s90 00:08:38.719 --> 00:08:44.800 Bring back those sections and then reason over that and then just generally for for longer horizon sessions. s91 00:08:44.800 --> 00:08:48.320 And when you want to skip it, of course, it' be something that fits in context. s92 00:08:48.320 --> 00:08:54.480 You want something that's low latency or the the model itself is as strong of a coder. s93 00:08:54.480 --> 00:09:00.640 And uh our friend Raymond here did some great perfor performance testing on the long chain of thought benchmark. s94 00:09:00.640 --> 00:09:07.680 Um I'll leave this link as a as a leave behind after, but just to give you a sense of how well it performs on some of these tasks. s95 00:09:07.680 --> 00:09:14.480 It's a meaningful jump overall from from 2.6 to 45.4% um accuracy on many of these tasks. s96 00:09:14.480 --> 00:09:18.720 And you can see it performs really well on things that are amanable to code. s97 00:09:18.720 --> 00:09:37.760 So logic puzzles and and chess and chemistry and things like that where it can dynamically write code bring in only the relevant part of the context compute that and then return the result where the main model is really just harvesting the results from from the subLM and try instead of trying to do that by itself. s98 00:09:38.160 --> 00:09:41.200 Um I put together a few just super simple examples. s99 00:09:41.200 --> 00:09:52.560 I mean, these are kind of they're somewhat unfair, I suppose, to to the base model, but it it makes the point that there are certain tasks that base models just aren't really fit to do themselves s100 00:09:52.560 --> 00:10:01.279 because they because they have to attend all these to different tokens at once in the context window where you need or want to use some type of coding approach to that. s101 00:10:01.279 --> 00:10:16.800 So in this random example, summing 12 numbers that are buried across 30,000 tokens, the LLM trying to figure all that out by itself and give you the answer isn't always going to work as well as something that you can write reg x4 or something similar. s102 00:10:16.800 --> 00:10:28.320 And then the same same sort of thing particularly for data frames and we'll walk through a brief example here where because the LLM can interact with the data frame within the ripple. s103 00:10:28.320 --> 00:10:39.360 It just has a much better understanding of the content and can iterate through that much more quickly than having to pass tool calls back and forth in terms of like JSON strings and and that sort of thing. s104 00:10:39.360 --> 00:10:44.240 Um, and then I threw this in there in terms of running the same experiments with a coding agent. s105 00:10:44.240 --> 00:10:46.720 Now, I didn't look into this too deeply. s106 00:10:46.720 --> 00:10:56.560 There's probably some unfair math going on here, but you can see that it was totally bloated in terms of the way that cloud code tried to um tried to solve these tasks. s107 00:10:56.560 --> 00:11:06.720 So, there's more work to be done there, of course, in terms of like running experiments to compare base models versus RLMs versus something like a coding agent. s108 00:11:06.720 --> 00:11:10.560 But there's just for certain tasks for like production workloads. s109 00:11:10.560 --> 00:11:16.399 My sense is you probably don't want to just do cloud-p your prompt and like hope for a good result. s110 00:11:16.399 --> 00:11:27.760 Like you want more of a structured approach to your inputs, your outputs and you want a defined pipeline for doing so which reduces your cost, it reduces your complexity, reduces your bloat, all that sort of thing. s111 00:11:27.760 --> 00:11:32.000 Um where RLMs can can shine. s112 00:11:32.079 --> 00:11:38.560 So in the real world there are a bunch of different open source libraries that implement RLMs at some level. s113 00:11:38.560 --> 00:11:50.240 Some of them are more RLM focused uh like a predict RLM would be a good example of that versus others are kind of just integrating it into the broader approach or the broader framework. s114 00:11:50.240 --> 00:11:55.680 DSPI obviously uh there's axe which is really interesting work uh that's being done there. s115 00:11:55.680 --> 00:11:58.800 Predict RLM is more focused on like knowledge work. s116 00:11:58.800 --> 00:12:04.000 So it works with spreadsheets and PDFs and that sort of thing and then fast RLM. s117 00:12:04.000 --> 00:12:09.760 And then there's a tweet yesterday from this guy Sam Hogan um where who runs inference.net. s118 00:12:09.760 --> 00:12:28.000 He's using an RLM to basically run and extract um insights from your particular um production workload traces so that they can see what makes sense to defer off to something like a GLM 5.2 too and do that iteratively and automatically as your traffic goes through. s119 00:12:28.000 --> 00:12:33.040 So point being, you don't have to worry about context engineering. s120 00:12:33.040 --> 00:12:36.880 You can kind of just throw the RLM at it and have it figure it out. s121 00:12:36.880 --> 00:12:43.680 Um, I only have five minutes left, so we won't go through this whole example and I'll I'll skip to some of the traces because that's probably the most interesting. s122 00:12:43.680 --> 00:12:54.320 Um, but this is all you would really need to do in terms of a simple, in this case it's like a a cohort retention analysis, something that you might give to a data scientist. s123 00:12:54.320 --> 00:13:02.240 But this concept of applying an RLM to a complex data structure like a data frame becomes very easy to do. s124 00:13:02.240 --> 00:13:06.720 This is all the code you need to do it where I'm feeding in three different data frames. s125 00:13:06.720 --> 00:13:09.600 I'm saying these are the sorts of things you need to look for. s126 00:13:09.600 --> 00:13:11.839 These are the output types that I want. s127 00:13:11.839 --> 00:13:14.959 and then just let the RLM go on it. s128 00:13:14.959 --> 00:13:18.079 And I'll I'll show you some of the traces. s129 00:13:18.079 --> 00:13:23.600 Um, and so it has its own ripple where it can interact with those data frames. s130 00:13:23.600 --> 00:13:25.040 And you can see it reasoning through. s131 00:13:25.040 --> 00:13:26.480 Okay, first I need to do this. s132 00:13:26.480 --> 00:13:27.920 It's writing the code. s133 00:13:27.920 --> 00:13:34.959 And because it's li it's living in the ripple with the dataf frame, you don't have this additional bloat of the tool calls back and forth. s134 00:13:34.959 --> 00:13:40.240 It's actually interacting directly with the dataf frame as if it was typing in its own Jupyter notebook. s135 00:13:40.240 --> 00:13:44.480 And there there's significant advantages for for doing so. s136 00:13:44.480 --> 00:13:48.240 And so you can see the sorts of outputs that it gets as a result. s137 00:13:48.240 --> 00:13:50.560 And it by itself will iterate. s138 00:13:50.560 --> 00:13:55.680 And in this case it didn't, but it has the option to defer to subLM to do okay. s139 00:13:55.680 --> 00:14:05.040 And now I have this big whatever this big subset of the data sublm go off and do this analysis give me the result and it can do that iteratively over time. s140 00:14:05.040 --> 00:14:15.920 Uh but the point is that the LM is directly interacting with the data frame in its ripple um and kind of iterating through the results. s141 00:14:15.920 --> 00:14:20.959 And so this u this platform compound is RLM and DSPI native. s142 00:14:20.959 --> 00:14:23.920 So it gives you this really nice breakdown of the reasoning. s143 00:14:23.920 --> 00:14:34.240 It separates out the code that's being generated and ultimately you can see uh the final output which is here where it's formatting. s144 00:14:34.240 --> 00:14:35.920 Okay, here are the key findings that I have. s145 00:14:35.920 --> 00:14:37.600 Here are the recommendations. s146 00:14:37.600 --> 00:14:45.760 And then you have this final submit which is the final answer that gives you the the typed um outputs that you had defined up front. s147 00:14:45.760 --> 00:14:49.920 And the key thing here is that the LLM itself is deciding when to stop. s148 00:14:49.920 --> 00:14:53.519 So you have this you have a variable of max iteration. s149 00:14:53.519 --> 00:14:58.800 So you can just you can decide whether you want it to have a maximum of 10 or 100 or whatever it is. s150 00:14:58.800 --> 00:15:09.120 But it will by itself explore the data, understand what needs to happen and then when it it itself is comfortable, it can run submit and give you the final output. s151 00:15:09.120 --> 00:15:12.320 Again, being bitter or less impilled, this will get better over time. s152 00:15:12.320 --> 00:15:16.959 You can kind of just defer everything and it will figure out what to do. s153 00:15:16.959 --> 00:15:23.199 And so the hope would be you don't have to I mean we're already you know whatever this is 20 lines of code or something. s154 00:15:23.199 --> 00:15:29.279 Um, but you can you can see a world where you can continue to go up levels of abstraction. s155 00:15:29.279 --> 00:15:36.000 As long as you can define what your objective is and what you want it to do, the the model will kind of figure out the rest. s156 00:15:36.000 --> 00:15:43.519 Uh, so we just walk through a bunch of this, but um these are the different steps that it took in this example in the code that it wrote. s157 00:15:43.519 --> 00:15:48.959 Um, and then I'll just breeze through a few real world case studies and where it's actually being used. s158 00:15:48.959 --> 00:15:51.120 So I mentioned predict rm before. s159 00:15:51.120 --> 00:16:01.759 So the company trampoline AI I think it is they're doing really interesting work in applying RLMs u like I mentioned before for different pieces of knowledge work. s160 00:16:01.759 --> 00:16:06.720 So natively interacting with PDFs and and spreadsheets and that sort of thing. s161 00:16:06.720 --> 00:16:17.120 So in this relatively simple example okay I have a bunch of I have a directory of invoices that I need to create one consolidated inventory out of. s162 00:16:17.120 --> 00:16:19.839 As we all know invoices can be complicated. s163 00:16:19.839 --> 00:16:23.839 that can be very long, that can kind of be all over the place. s164 00:16:23.839 --> 00:16:31.040 To do that today without RLMs or this sort of like framework gets very complicated very quickly. s165 00:16:31.040 --> 00:16:33.680 I have a lot of battle scars to to prove it. s166 00:16:33.680 --> 00:16:47.600 Um, but with something like an RLM, you you don't need to worry as much about, okay, if I have a 200page invoice or contract or whatever it is, you can let the RLM just churn through all of that and give you the result instead of having to worry about chunking s167 00:16:47.600 --> 00:16:57.199 and embedding maybe and doing all these different strategies to try and get around the context window management that we've all had to to do previously. s168 00:16:57.360 --> 00:17:07.839 Um, so it allows you, the point there is that you can focus on the abstractions and what you actually want to do instead of the context engineering itself, which I think is a really helpful s169 00:17:07.839 --> 00:17:10.160 um, helpful output of all of this. s170 00:17:10.160 --> 00:17:28.160 And an interesting tidbit for all the DSPI fans in the room, predict RLM uses DSPI to determine the schemas between the main LM and the subLM calls, which I personally think is is a nice feature because you have a lot more readability and maintainability. s171 00:17:28.160 --> 00:17:38.320 So you understand exactly what the model is trying to achieve and the model can be much more precise and prescriptive about the types of data that it's looking for from the subLM. s172 00:17:38.320 --> 00:17:48.000 And I would want to do some experiments to test this out, but I would think that this would improve performance for cheaper models like a a Quinn or some of the other ones because s173 00:17:48.000 --> 00:17:52.000 you're specifying the inputs and outputs and you're enforcing those types coming back. s174 00:17:52.000 --> 00:18:03.200 And so you get all the benefits of the RLM being able to churn through all this information, but you have a lot more of the structure in between where when it's handing off to a sublm, s175 00:18:03.200 --> 00:18:07.120 it it enforces some of the uh some of those schemas. s176 00:18:07.120 --> 00:18:12.960 This is an example from um an AWS engineer from a couple days ago. s177 00:18:12.960 --> 00:18:21.840 We were just kind of playing around with it, but I just thought it was a nice example of you can kind of just throw arbitrary data at RLM. s178 00:18:21.840 --> 00:18:30.240 In this case, it was a bunch of log data um to surface some interesting uh results and he he found it useful. s179 00:18:30.240 --> 00:18:41.440 Um there's a a project called Halo which uses an RLM to look at traces of um of different t uh agent tasks. s180 00:18:41.440 --> 00:18:55.840 And basically the promise of Halo is that instead of optimizing a particular like workflow or DSPI uh or or other framework uh like structure itself, it's it's actually iterating on the harness. s181 00:18:55.840 --> 00:19:07.280 So it's like a meta abstraction almost or meta optimization of the harness itself and it uses an RLM because as we all know tracing can get very long and and complicated. s182 00:19:07.280 --> 00:19:18.880 So the RLM can not only take in all that context but also leverage the um the structure of those traces to to recommend a better um a better harness. s183 00:19:18.880 --> 00:19:22.240 And then this last one uh this is all the code you need. s184 00:19:22.240 --> 00:19:24.080 I ran this little experiment. s185 00:19:24.080 --> 00:19:32.799 There's a an intentionally vulnerable application called uh it's from OASP, but basically there's a it's a web app with a bunch of vulnerabilities in it. s186 00:19:32.799 --> 00:19:42.799 This is all the code you need on the right hand side to run basically an agent to run through whatever it is 500,000 lines of code to generate some type of security report. s187 00:19:42.799 --> 00:19:48.400 That's just an arbitrary example, but the point is you don't need a lot of context engineering. s188 00:19:48.400 --> 00:19:52.240 You don't need a lot of structure around it to achieve what you want to do. s189 00:19:52.240 --> 00:19:58.880 And so you can feed in an arbitrary uh size codebase into this and get some type of insights out. s190 00:19:58.880 --> 00:20:02.559 So you can imagine that being applied to other areas as well. s191 00:20:02.559 --> 00:20:07.520 Um so I know I I rushed through everything a little bit but I'm happy to answer questions afterwards. s192 00:20:07.520 --> 00:20:09.919 Uh the I'll leave you leave you with this. s193 00:20:09.919 --> 00:20:17.919 The biggest promise I see here is just imagine a world where the models are actually post-trained and actually like RLM aware. s194 00:20:17.919 --> 00:20:27.919 I think things will get pretty crazy pretty quick when they actually know how to use and kind of take advantage of the RLM methodology natively. s195 00:20:27.919 --> 00:20:31.440 So, thank you so much for your time.