WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=oVsEddfhdxc NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/oVsEddfhdxc.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:12.559 --> 00:00:14.400 Hey um hi everyone. s2 00:00:14.400 --> 00:00:16.240 Uh thanks for being here. s3 00:00:16.240 --> 00:00:28.880 Uh yeah, I'm super happy today to talk about uh automated eye research and uh especially uh all those like frontier model uh perform at uh automated eye research task. s4 00:00:28.880 --> 00:00:30.240 Um so I'm Elie. s5 00:00:30.240 --> 00:00:36.960 I worked at Prime Intellect as a research engineer and uh yeah I will go through our work on on this subject. s6 00:00:36.960 --> 00:00:45.600 So first I want to basically explain a bit why we are doing that and why we think it's super important to do that in the open. s7 00:00:45.600 --> 00:00:59.199 Um so first uh I think we we all agree that uh we've heard about like big labs saying that this bad thing called recursive self-improvement is coming very soon. s8 00:00:59.199 --> 00:01:06.400 Uh so recursive self-improvement is like model training models uh without uh human intervention basically. s9 00:01:06.400 --> 00:01:13.360 Um but uh we don't have any benchmark to basically quantify if this is true or not right. s10 00:01:13.360 --> 00:01:23.840 Uh and even less we don't have like a third party benchmark by non- big labs to to to see if it's something coming soon or not. s11 00:01:23.840 --> 00:01:36.640 And the other part is that we think that uh it's super important to understand all those model uh do research because we think that a lot of the scientific research that will come into the coming years s12 00:01:36.640 --> 00:01:39.520 uh will be based also on AI tools. s13 00:01:39.520 --> 00:01:45.360 So it's super important to understand how those model do research not just only AI research. s14 00:01:45.360 --> 00:01:53.360 So we try to build kind of this environment to test the capabilities of the model to do so. s15 00:01:53.360 --> 00:02:09.360 So it all started with uh Andre Karpati uh that's basically had fun by doing this video where he trained uh GPT2 from scratch in like 90 minutes like GPT2 training takes like weeks s16 00:02:09.360 --> 00:02:13.440 and uh no in two years ago I think it only took like 90 minutes. s17 00:02:13.440 --> 00:02:18.560 So what does it mean to reprod reproduce uh GPT2 in 90 minutes? s18 00:02:18.560 --> 00:02:22.319 It means that in 90 minutes you achieve this target loss. s19 00:02:22.319 --> 00:02:34.319 Um and yeah and that's at this point when you have the same loss than GPT2 you consider that your model is somewhat of equal performance. s20 00:02:34.319 --> 00:02:47.599 Um then what happened is that the community took this repo uh this GitHub repo and create another one called moed nano GPT and this effort was leaded by someone called Keller Jordan. s21 00:02:47.599 --> 00:03:00.640 And what happened is that they basically took this 90 minutes then 45 minutes and then now we can train like GPT2 validation loss model in less than two minutes which is honestly s22 00:03:00.640 --> 00:03:04.000 crazy and it took like two years to to achieve this. s23 00:03:04.000 --> 00:03:27.920 So it's a very strong benchmark where uh a lot of very talented researcher iterated on um yeah so we decided to take this environment of speedun so it's kind of a game so the goal of the game is to achieve this loss in the fewest in the shortest amount of time so this is the nano GPT1 s24 00:03:27.920 --> 00:03:38.239 and you can uh you don't have almost any constraints the only constraint that you shots that you need to use the same validation and training data, right? s25 00:03:38.239 --> 00:03:49.840 Um there is a new speedrun called the optimizer speedrun that was released uh a few months ago and here it's slightly different because uh you can only change the optimizer s26 00:03:49.840 --> 00:03:51.360 related parameters. s27 00:03:51.360 --> 00:04:06.959 So for instance nano GPT you can change the architecture uh do uh attention whatever uh optimizer sp you can only change like Adam to m shampoo or whatever optimizer s28 00:04:06.959 --> 00:04:20.959 is your favorite um yeah and so this is a bit more researchy because uh it's less about optimizing the program to be as fast as possible but more like finding the best method possible. s29 00:04:20.959 --> 00:04:26.400 no matter the the the time you put into the computer, right? s30 00:04:26.400 --> 00:04:33.120 So, um yeah, why take speedrun as an environment for automated AI research? s31 00:04:33.120 --> 00:04:36.240 First, uh we think that it's a good evaluation. s32 00:04:36.240 --> 00:04:37.680 We'll see later why. s33 00:04:37.680 --> 00:04:40.800 Uh and this is kind of the main focus of this talk. s34 00:04:40.800 --> 00:04:47.840 But we also think it's probably a good training environment because uh it's a way to give the model a reward. s35 00:04:47.840 --> 00:04:59.199 So the reward is positive if the model bit the speed run and beat the last record sorry and the reward is zero or negative if it didn't manage to to do it. s36 00:04:59.199 --> 00:05:01.520 So it's a good uh environment to train model. s37 00:05:01.520 --> 00:05:08.080 It's also quite fast like as you see uh previous record were around 2 minutes for the optimizer one. s38 00:05:08.080 --> 00:05:20.160 uh each run take about like 15 to 20 minutes and uh yeah and there is like clear rules basically and we also think it's like a good environment to make discovery s39 00:05:20.160 --> 00:05:28.080 so like kind of breakthrough in our research because uh there is those clear rule that you can verify or not. s40 00:05:28.080 --> 00:05:29.759 Um yeah. s41 00:05:29.759 --> 00:05:32.160 So yeah. s42 00:05:32.160 --> 00:05:44.479 Um so what we did uh so the release was like about two months ago and uh there was this optimizer speedrun and we decided to basically compete with the community s43 00:05:44.479 --> 00:05:46.639 by launching two AI agents. s44 00:05:46.639 --> 00:05:48.400 So Codex and Cloud Code. s45 00:05:48.400 --> 00:05:56.800 Codex was like GPT 5.5 with XI and uh cloud code was Opus 4.8 with XI. s46 00:05:56.800 --> 00:06:04.720 Um and yeah, we decided to basically let the agent free on our cluster uh and uh and just iterate on it. s47 00:06:04.720 --> 00:06:10.880 So we have like V1, V2, V3 is just basically us stopping the agent and then restarting. s48 00:06:10.880 --> 00:06:18.639 V3 uh was like one or two day before the release because we saw that our agents no longer have the best record. s49 00:06:18.639 --> 00:06:28.560 So we were like okay take all the the human uh record in the last few week and just try to to improve upon it and and and it worked. s50 00:06:28.560 --> 00:06:28.880 Yeah. s51 00:06:28.880 --> 00:06:36.720 And we also have this novelty track where the goal is to uh beat the record with only novel ideas. s52 00:06:36.720 --> 00:06:43.759 Um and we'll see that this this was more complex for the the models. s53 00:06:43.759 --> 00:06:46.240 So our RS is very simple. s54 00:06:46.240 --> 00:06:51.440 Honestly, we could have just replaced it with slashgo, but they there was no SLG goal at the time. s55 00:06:51.440 --> 00:06:53.520 So, we made our own goalm. s56 00:06:53.520 --> 00:06:57.840 It's actually quite fun that we choose the same name and we had the goal. s57 00:06:57.840 --> 00:07:11.120 MD and kind of agents that MD that define the rules and we let the agent propose ids and then it can submit a job with sbatch on our slum cluster s58 00:07:11.120 --> 00:07:22.000 and uh basically the way it works is that it can submit on nodes that are available but only under a certain permission which means that if someone want to use this node s59 00:07:22.000 --> 00:07:26.560 uh the model just like cancel the job it's called preemptable permission. s60 00:07:26.560 --> 00:07:32.560 So yeah, then it measure the it read basically the training logs then decide if it's a record or not. s61 00:07:32.560 --> 00:07:40.960 To validate a record you need to basically pass a statistical threshold to make sure that it's just not see the optimization and is just not random. s62 00:07:40.960 --> 00:07:42.560 Right? s63 00:07:42.560 --> 00:07:45.360 So yeah a few results from this experiment. s64 00:07:45.360 --> 00:07:58.800 The first one that was honestly very painful to work with is that code clothes code keep stopping every nine or 10 hours and basically said yeah I cannot improve the record it's too hard for me s65 00:07:58.800 --> 00:08:11.199 there is no way to to go beyond it and then I was just like okay continue explore new direction hey just go again for 10 hours and then say yeah I cannot s66 00:08:11.199 --> 00:08:12.319 beat the recall and so on. s67 00:08:12.319 --> 00:08:24.319 So basically onethird of the time the cloud code agent was idle because I had no way to basically monitor it and codeex totally the opposite just worked for all the s68 00:08:24.319 --> 00:08:31.520 all the time and uh yeah almost never idle never asked for question and and and very impressive in that way. s69 00:08:31.520 --> 00:08:44.240 Um we also give the option for the model to basically write uh a bunch of stuff into what we call a scratch pad which is basically the active memory of the model. s70 00:08:44.240 --> 00:08:50.640 Uh we observe that basically codeex writes a lot on the scratch patch. s71 00:08:50.640 --> 00:08:55.600 So each plot that I will show are kind of normalized by the number of active order. s72 00:08:55.600 --> 00:09:02.160 So this is not only about codex working more it's it's really different behavior. s73 00:09:02.160 --> 00:09:14.320 So yeah, you see that uh writes a lot more to to this scratch pad to this memory and uh the shape of the like the the I don't know the tone of the the each file s74 00:09:14.320 --> 00:09:27.920 was also super different like CL was super excited about getting new record with a bunch of emoji and so on and CEX was just like here is what I do here is the decision I take what I will do next like super s75 00:09:27.920 --> 00:09:39.279 robotic kind of um Yeah, we also have this plot where basically we saw that codex was spawning much more sub aents than cloud. s76 00:09:39.279 --> 00:09:42.640 Uh we saw that codex burn much more token than code. s77 00:09:42.640 --> 00:09:55.760 So I think in total it was like kind billion of token but it's like there is obviously this input tok uh input caching that make it it's not like one billion output token. s78 00:09:55.760 --> 00:10:08.880 Uh so yeah we also see that codex did a lot of compaction because it only had like 250k context window and cloud only do it like one per hour and codex is more like s79 00:10:08.880 --> 00:10:18.320 no it's even less than one power for I mean one for the full run for code and codex was like one uh was 20 every one hour. s80 00:10:18.320 --> 00:10:23.440 So yeah um yeah here is the main results. s81 00:10:23.440 --> 00:10:41.760 So what this plot shows is that basically we so in in white you see that the human record progression right and in red you see cloud I mean it's supposed to be orange but whatever and in blue uh you see codeex right s82 00:10:41.760 --> 00:10:53.360 and you see that at almost every time uh cloud and codex are better than the human record and code is super good at the beginning very very fast to achieve very good s83 00:10:53.360 --> 00:11:05.360 Um yeah and one thing that is super important is that the model have the ability to basically fetch the human records at any time and that's what codex did that's what cloud did sorry s84 00:11:05.360 --> 00:11:18.720 because when I restarted it it basically fetch the new record from human and improve upon it um yeah so the result is that uh I think at the time the best uh record was like s85 00:11:18.720 --> 00:11:30.000 uh 2,990 step and we beat it by like uh uh 50 or 60 step for code and codeex was like 20 step above. s86 00:11:30.000 --> 00:11:38.000 So I think it's both impressive and and yeah um so we so this is like not released yet. s87 00:11:38.000 --> 00:11:47.120 This is something that we are working on currently and basically the idea is that this is a cool experiment to do but it lack of structured right. s88 00:11:47.120 --> 00:12:00.399 uh if you want to do a real benchmark, you want to do multiple seed, you want to do uh yeah proper uh thing where you you you you basically put all the model and earnestness in the same condition, right? s89 00:12:00.399 --> 00:12:15.120 So this is what we are working on right now and basically um the idea is to do three different track uh one without any access to really like measure the capability of the models to do AI research based on s90 00:12:15.120 --> 00:12:22.320 only the model weight knowledge one with only archive paper and one with like full access. s91 00:12:22.320 --> 00:12:27.839 So it also have access to the the like the latest record by human. s92 00:12:27.839 --> 00:12:39.839 And for this we plan to do both uh the nano GPT track one which is the original one and the optimizer speedun where we we only launch uh we only constrain the the optimizer s93 00:12:39.839 --> 00:12:42.560 to be to be novel basically. s94 00:12:42.560 --> 00:12:47.839 Um yeah so I will present some result on the optimizer speedrun. s95 00:12:47.839 --> 00:12:49.920 Uh this is basically what we got. s96 00:12:49.920 --> 00:13:03.360 So we let the agent iterate for six day almost five days let's say and we see that uh codeex k and clothes uh are super effective so for GLM s97 00:13:03.360 --> 00:13:17.200 this is not finished run right so the model is actually still iterating on the cluster right now but we see that cloud is once again very good at it and we see that surprisingly Kim is also very competitive s98 00:13:17.200 --> 00:13:24.480 and kind of have this breakthrough on day four where he kind of beat Codex with a new record, right? s99 00:13:24.480 --> 00:13:37.519 It's also interesting to see that uh Claude is much more like progressive in the way it improved the record and Kim has really this step function where I kind of do a breakthrough and so on. s100 00:13:37.519 --> 00:13:48.720 Uh so this is an interesting plot because I mean six day is quite a lot for anal uh uh but you you can change this uh axis by also the number of output token s101 00:13:48.720 --> 00:14:00.720 and then kind of tell a different story because in max mode consumes so much more token than codeex and kimmy and you also see that Kimmy is actually super efficient s102 00:14:00.720 --> 00:14:04.000 uh for the number of token that uh it uses. s103 00:14:04.000 --> 00:14:06.639 So it's scheming K2.7 code. s104 00:14:06.639 --> 00:14:16.079 Um so yeah uh we also see that they have a different approach to uh using the literature and papers. s105 00:14:16.079 --> 00:14:26.720 Um so for instance like code is doing a lot of search on papers and actually include found a paper that no other model found and it actually lead to s106 00:14:26.720 --> 00:14:27.600 the best record. s107 00:14:27.600 --> 00:14:49.519 So it's kind of funny and uh yeah um one of the main issue of all of this is that uh when I when I launched this this agent and I think that's something important that I want you to to kind of uh remember for this co this talk is that when I launched this these different agents I s108 00:14:49.519 --> 00:14:58.240 was expecting them to come up with some crazy ideas on optimizer that's like no one of discover but honestly it wasn't the case. s109 00:14:58.240 --> 00:15:03.519 Uh they did some clever trick where basically they combine different papers. s110 00:15:03.519 --> 00:15:15.680 uh they kind of do plus one improvement over a bunch of method but there was really like no novel optimizer or mechanism that was uh coming from those model and I think that's s111 00:15:15.680 --> 00:15:31.519 kind of telling that even on something that is not simple but I'd say that it's kind of accessible for people right for like human researcher uh spending like days and weeks for the the model like s112 00:15:31.519 --> 00:15:35.760 cannot like find new uh optimizer and mechanism. s113 00:15:35.760 --> 00:15:53.920 So we believe that there is a way to basically make it more um make it better for discovery instead of evaluation and this is coming from uh this is very inspired from alpha evolve by Google and also a bunch of papers that have been released since then. s114 00:15:53.920 --> 00:16:00.399 It's kind of this multi- aent system that interact together uh bunch of generator. s115 00:16:00.399 --> 00:16:06.480 You have closed model but you also have open source model here that are super effective for the cost right. s116 00:16:06.480 --> 00:16:16.320 Uh they can suggest ideas then you run the speedrun so you get the reward then you have a judge that basically give a quality feedback can also be like s117 00:16:16.320 --> 00:16:18.399 the judge also have this taste. s118 00:16:18.399 --> 00:16:30.880 you can kind of have like the judge have a taste about the the method if it's good or not uh if it's outside the loop and then you can uh basically decide s119 00:16:30.880 --> 00:16:37.920 which method you want to scale to a larger number of parameters and number of token. s120 00:16:37.920 --> 00:16:48.800 Um so this is kind of the scale part of the speedrun because some a lot of method in the the speedrun community uh people are often saying that they doesn't work at large scale. s121 00:16:48.800 --> 00:16:53.680 So I think it's very important to also put scale elements in this loop. s122 00:16:53.680 --> 00:17:05.600 Uh and I think also that uh human are super useful here to basically judge the ID of agents kind of steer them in the right direction and so on. s123 00:17:05.600 --> 00:17:18.240 Um yeah so we didn't try it yet I mean we are kind of trying it right now and uh we hope that this will lead to to to new discovery in AI research at least and also a way s124 00:17:18.240 --> 00:17:32.080 is that you can define multiple speedrun so this is the next slide if you like it's from safe bank slides but if you if you don't have the reference good for you means that that you're not too online s125 00:17:32.080 --> 00:17:44.000 uh but the idea is that uh by changing the object objective and the constraints of the speedrun you can basically create a lot of diversity and constrain the model to go into a certain direction s126 00:17:44.000 --> 00:17:47.600 and uh yeah and make those discovery. s127 00:17:47.760 --> 00:17:51.120 So uh at HIM in Ind. s128 00:17:52.000 --> 00:18:02.000 Uh there is a bunch of stuff here that we I mean most of it we didn't release yet but we are working on GPU sandboxing to allow model to iterate into s129 00:18:02.000 --> 00:18:05.360 sandbox because you need GPU sandbox for this kind of stuff. s130 00:18:05.360 --> 00:18:12.960 We are working on our own agents that are very efficient for like RM framework. s131 00:18:12.960 --> 00:18:17.760 So it means like you have a file system and you can write information read from it. s132 00:18:17.760 --> 00:18:21.120 Uh and you also do like this programmatic tool coding thing. s133 00:18:21.120 --> 00:18:26.320 We also training a model to be good at it on top of like open source model. s134 00:18:26.320 --> 00:18:38.400 And uh the thing that we already released is that we have those set of liberation and product called verifier primaril training where you can basically train evaluate any environments on s135 00:18:38.400 --> 00:18:47.919 any RS and the model that you can train can be like GNM 5.2 too which is very big and and yeah we have like we work a lot on making those li very efficient s136 00:18:47.919 --> 00:18:51.679 to to ship the best quality for for our clients. s137 00:18:51.679 --> 00:18:52.559 Yeah. s138 00:18:52.559 --> 00:18:56.640 Uh I mean yeah super excited about this domain. s139 00:18:56.640 --> 00:19:06.720 Once again I think it's super important to have uh a part of like this recursive self-improvement to happen in the open because there is actually a lot of people s140 00:19:06.720 --> 00:19:08.720 working that are not on big labs. s141 00:19:08.720 --> 00:19:16.400 So you need to basically uh yeah make it easy for people to understand all those model work to do research and so on. s142 00:19:16.400 --> 00:19:21.440 So that's kind of our goal and uh yeah thanks a lot.