WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=iqloyWCGYQQ NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/iqloyWCGYQQ.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:01.309 --> 00:00:03.309 [music] s2 00:00:12.639 --> 00:00:13.840 Hi everyone. s3 00:00:13.840 --> 00:00:15.440 Uh my name is Partasawa. s4 00:00:15.440 --> 00:00:18.800 I'm a PhD student at UC Berkeley. s5 00:00:18.800 --> 00:00:26.240 We're all at these AI and machine learning conferences, but everyone is just talking about how smart these agents are. s6 00:00:26.240 --> 00:00:30.720 We're not actually talking about learning ability or how much these agents learn. s7 00:00:30.720 --> 00:00:34.559 And so today I want to talk about that. s8 00:00:35.520 --> 00:00:38.160 You've seen the way we evaluate language models today. s9 00:00:38.160 --> 00:00:43.920 Every time there's a new model release, we see charts and graphs that look something like this, right? s10 00:00:43.920 --> 00:00:53.280 The way we evaluate these language models is we ask them to do one task and then completely independently we ask them to do another task and then another. s11 00:00:53.280 --> 00:00:56.960 And then we repeat this across a variety of different benchmarks, right? s12 00:00:56.960 --> 00:00:59.760 And so you've probably seen a variety of all of these. s13 00:00:59.760 --> 00:01:08.400 In the end, those benchmarks in aggregate when we take all of those independent evaluation scores give us leaderboards that look like this. s14 00:01:08.400 --> 00:01:21.840 Now, the challenge here is that if you look at the way we actually come to these single independent scores, what we've done is we've kind of told the models, imagine that every time you do something, you completely forget your memory. s15 00:01:21.840 --> 00:01:22.240 Okay? s16 00:01:22.240 --> 00:01:23.520 Like imagine if your life was like that. s17 00:01:23.520 --> 00:01:26.799 You're restarting from scratch every single time you do anything. s18 00:01:26.799 --> 00:01:31.119 That's the premise under which we're evaluating language models today. s19 00:01:31.119 --> 00:01:34.079 I'm going to argue that it's not necessarily what it should look like. s20 00:01:34.079 --> 00:01:43.600 If you were to look at learning ability, you might see charts where across different instances, performance doesn't look like a scattered line, but it looks like as a function of prior experience, s21 00:01:43.600 --> 00:01:49.520 models are actually able to improve their performance in contrast to what they've done in the past. s22 00:01:49.520 --> 00:01:51.119 Let's take a step back. s23 00:01:51.119 --> 00:01:53.759 What is continual learning? s24 00:01:53.759 --> 00:01:59.040 Continual learning to me is sample efficient online learning that is stable over long horizons. s25 00:01:59.040 --> 00:02:12.160 It's a challenging problem because you have to deal with both your ability to retain prior information without forgetting while simultaneously being able to update from new information and adjust your priors to new new data you're receiving over time. s26 00:02:12.160 --> 00:02:27.120 In language models specifically, you know, the way we train language models is we take a bunch of data we want to train them on and we kind of put them in a box in some offline training process and then maybe for a few weeks or something we train the model and then we extract a frozen checkpoint, s27 00:02:27.120 --> 00:02:27.360 right? s28 00:02:27.360 --> 00:02:29.760 And this frozen checkpoint is what's deployed to the world. s29 00:02:29.760 --> 00:02:32.959 The weights don't really change after that in today's paradigm. s30 00:02:32.959 --> 00:02:38.640 And continual learning is all about trying to change that, trying to enable mechanisms for models to learn over time. s31 00:02:38.640 --> 00:02:43.680 So in language models, people have looked to approaches like in context learning. s32 00:02:43.680 --> 00:02:48.400 Put more information in the model's context and the model can simulate some amount of learning in context. s33 00:02:48.400 --> 00:02:51.200 People have looked towards external memory stores, right? s34 00:02:51.200 --> 00:02:59.920 You could externalize some of your memory to a notepad that you then get to refer to every single time or a key value store or your choice of external memory. s35 00:02:59.920 --> 00:03:03.120 Or people look towards the parametric approaches. s36 00:03:03.120 --> 00:03:11.680 What would it actually look like if we had ways to update the model's weights online such that it could improve and learn from the new information? s37 00:03:11.680 --> 00:03:18.000 Now, today I'm not going to make a case for which one of these can solve continual learning, though I do have an opinion and I'll share it at the end. s38 00:03:18.000 --> 00:03:23.120 But what I am going to make a case for is that we're actually not evaluating it at all today. s39 00:03:23.120 --> 00:03:33.920 And the question that we need to ask ourselves is that if continual learning doesn't look like point capabilities, we need to be measuring it the right way to optimize for the right objective as a field. s40 00:03:33.920 --> 00:03:35.120 So let's get into it, right? s41 00:03:35.120 --> 00:03:39.040 What does continual learning kind of look like today when you read the literature? s42 00:03:39.040 --> 00:03:41.840 To be clear, continual learning by itself is not a new idea. s43 00:03:41.840 --> 00:03:45.519 Continual learning for neural networks has been around for decades, right? s44 00:03:45.519 --> 00:03:50.879 There have been famous problems in catastrophic forgetting that people have spent years trying to solve. s45 00:03:50.879 --> 00:04:06.000 But with language and you know an example of that is some of these sorts of papers where you'll see graphs like this where you train the models on a particular task distribution task A then you train it on the task B distribution and then you train it on task C. And the objective you're looking for is the s46 00:04:06.000 --> 00:04:12.400 does the model's performance not degrade on prior tasks while it's still able to learn the new tasks. s47 00:04:12.400 --> 00:04:17.199 And people have taken this similar methodology and applied it to language models too. s48 00:04:17.199 --> 00:04:17.440 Right? s49 00:04:17.440 --> 00:04:23.520 In some of the more recent papers we've seen, you're you're starting to see similar graphs of you train on one task and you train on the next. s50 00:04:23.520 --> 00:04:36.960 There's also a different class of evaluations for language models that look more like a factual recall sort of test where over these extremely long horizons, can models remember particular parts of the conversation and improve from that. s51 00:04:36.960 --> 00:04:42.880 The case I'm going to make though is that these sorts of evaluations aren't actually sufficient for continual learning. s52 00:04:42.880 --> 00:04:46.560 Okay, they don't consider sample efficiency a first order requirement. s53 00:04:46.560 --> 00:04:48.880 They don't necessarily always measure learning. s54 00:04:48.880 --> 00:04:52.800 And perhaps most importantly, they won't work for frontier language models. s55 00:04:52.800 --> 00:05:00.479 And the reason being that frontier language models are pre-trained on vast distributions of the entire internet or economically valuable tasks we care about. s56 00:05:00.479 --> 00:05:08.000 And so if you're expecting to see improvement on new tasks, it's it's actually really hard to come up with what those sorts of tasks should look like. s57 00:05:08.880 --> 00:05:13.759 So given all of that, what should a continual learning benchmark look like? s58 00:05:13.759 --> 00:05:20.320 The biggest question that I get when I when I talk about this with people is why can't I just chain existing benchmarks together? s59 00:05:20.320 --> 00:05:23.199 Like we've we've seen all of these Amy data sets. s60 00:05:23.199 --> 00:05:26.479 Why can't I just take my AM problems and solve them in a sequence? s61 00:05:26.479 --> 00:05:34.479 And the fundamental problem there is that benchmark instances are in traditional language model evaluation are designed to be independent. s62 00:05:34.479 --> 00:05:44.639 That means they don't have shared structure across tasks and as a result you can't meaningfully expect them to improve from earlier experience in future instances. s63 00:05:44.639 --> 00:05:51.120 And this is the fundamental problem that exists with any approach that tries to chain prior benchmarks together. s64 00:05:51.120 --> 00:06:01.520 In contrast, when we're designing continual learning benchmarks, we looked at three main design criteria as things that are kind of a fundamental requirement for evaluating continual learning. s65 00:06:01.520 --> 00:06:03.440 The first one here is headroom. s66 00:06:03.440 --> 00:06:05.199 And this kind of goes to my last point. s67 00:06:05.199 --> 00:06:08.800 Frontier language models are trained on almost everything in the internet. s68 00:06:08.800 --> 00:06:16.960 If you want to evaluate continual learning, you need some sort of task that actually requires online adaptation or learning from the language models. s69 00:06:16.960 --> 00:06:22.639 And that doesn't exist for a wide variety of the benchmarks where the models are just trained offline on that data. s70 00:06:22.639 --> 00:06:32.639 If the model can improve on your tasks by just training offline and not actually require any online learning, then it's not a good task for measuring continual learning. s71 00:06:32.639 --> 00:06:36.240 The second point that I'm going to talk about is shared structure. s72 00:06:36.240 --> 00:06:43.360 What this means is when going back to my point of pre prior language modeling benchmarks all being independent instances. s73 00:06:43.360 --> 00:06:52.240 If you want there to be an expectation of improvement in how you're evaluating continual learning, then there needs to exist some shared latent structure between the tasks. s74 00:06:52.240 --> 00:07:00.400 You can kind of think of this as some sort of shared latent in the environment that your models or agents are learning over time and they're seeking to exploit s75 00:07:00.400 --> 00:07:08.080 these these latent structures that exist in the environments to improve their performance on future tasks as a result of their prior information. s76 00:07:08.080 --> 00:07:11.759 And the third criteria that I'll talk about is a learning mechanism. s77 00:07:11.759 --> 00:07:19.360 And what this means is just there has to be some realistic expectation that the models are able to learn as a result of their prior experience. s78 00:07:19.360 --> 00:07:21.280 This could look like scalar reward. s79 00:07:21.280 --> 00:07:22.720 It could look like error messages. s80 00:07:22.720 --> 00:07:24.479 It could look like textual feedback. s81 00:07:24.479 --> 00:07:31.599 The point being, there needs to exist something in the environment that's giving agents signal to learn and improve on future tasks. s82 00:07:31.599 --> 00:07:36.639 Otherwise, it's not really a fair measure of continual learning. s83 00:07:37.440 --> 00:07:42.240 Let's talk a little bit more about the metrics before we get into examples. s84 00:07:42.319 --> 00:07:46.800 Obviously, there's a simple metric for a lot of tasks that we've seen recently, which is reward. s85 00:07:46.800 --> 00:07:48.800 And in our benchmark, it's no different. s86 00:07:48.800 --> 00:07:53.440 Every task defines a per instance reward metric where reward serves as a proxy for learning. s87 00:07:53.440 --> 00:07:54.720 Higher reward is better. s88 00:07:54.720 --> 00:07:56.400 You could think of this as efficiency. s89 00:07:56.400 --> 00:07:57.759 You could think of this as profit. s90 00:07:57.759 --> 00:08:00.560 You could think of this as how close you were to the actual predictions. s91 00:08:00.560 --> 00:08:05.120 There's some single unit of reward that operates on a per instance basis. s92 00:08:05.120 --> 00:08:13.919 And you might ask like why can't I just evaluate continual learning by looking at cumulative reward across these sequences of tasks we're constructing? s93 00:08:13.919 --> 00:08:22.080 And the answer is that total reward alone might confound [clears throat] continual learning ability with base model strength. s94 00:08:22.080 --> 00:08:25.039 Let's look at a diagram to kind of explain why this might work. s95 00:08:25.039 --> 00:08:27.520 Here we have two different continual learning systems. s96 00:08:27.520 --> 00:08:31.759 Okay, the the red one and the black one, the ones connected by the dotted lines. s97 00:08:31.759 --> 00:08:37.360 The gray system at the bottom is the black system except it wasn't allowed to maintain state across time. s98 00:08:37.360 --> 00:08:41.279 Okay, so that's what the black system looks like if its memory was wiped every single time. s99 00:08:41.279 --> 00:08:47.920 But if its memory wasn't wiped, if it had some mechanism of learning, then you can kind of see that it tends to improve over time. s100 00:08:47.920 --> 00:08:57.120 The challenge here is that if we looked at the cumulative reward metric alone, the red system would look like it was the best the best performing system on this task, right? s101 00:08:57.120 --> 00:09:00.959 It let's assume for a second that there is per room to do better and it hasn't saturated it. s102 00:09:00.959 --> 00:09:10.240 It has a higher cumulative reward, but you can kind of tell that in comparison to uh to the black system, it doesn't necessarily improve over the stateless baseline. s103 00:09:10.240 --> 00:09:12.560 It's just a better system to begin with. s104 00:09:12.560 --> 00:09:17.040 And this is one of the confounds we run into when we're trying to measure cumulative reward. s105 00:09:17.040 --> 00:09:22.480 To deal with this, we look at the gain metric in addition to reward. s106 00:09:22.480 --> 00:09:26.959 Gain refers to the difference between stateful reward and stateless reward. s107 00:09:26.959 --> 00:09:32.320 What that means practically in our benchmark is that for any system, we run it through the benchmark twice. s108 00:09:32.320 --> 00:09:37.440 Once in the normal way where it's allowed to maintain state across all of the instances in the benchmark. s109 00:09:37.440 --> 00:09:40.240 That might mean it's learning and updating its policy. s110 00:09:40.240 --> 00:09:42.000 It might mean that it's updating its notes. s111 00:09:42.000 --> 00:09:46.160 It might mean that it's just growing its context length, but it's allowed to maintain state. s112 00:09:46.160 --> 00:09:48.800 The second thing we run is a stateless baseline. s113 00:09:48.800 --> 00:09:57.519 What that means is we reset the system between every instance of a task so that the model isn't actually allowed to continually learn in any meaningful way. s114 00:09:57.519 --> 00:10:03.200 The difference between these two numbers then for any instance intuitively is kind of a question. s115 00:10:03.200 --> 00:10:13.440 It answers the question of how much did if we're looking at gain on task five, how much did my prior experience on the first four tasks actually lead to an improvement in my performance on the fifth task. s116 00:10:13.440 --> 00:10:19.760 It isolates out what your benefit of from actually learning was versus your base model's initial capability. s117 00:10:19.760 --> 00:10:24.640 To to be clear, I think reward, gain, and cost are all measured on Parto Frontiers. s118 00:10:24.640 --> 00:10:30.079 There isn't one single metric that I think defines continual learning because we still care about the base model strength. s119 00:10:30.079 --> 00:10:39.279 We still care about the ability to learn and we still care about the cost we're expending for these systems and so we try to measure everything on PTO Frontiers in this benchmark. s120 00:10:39.360 --> 00:10:43.920 Let's jump into an example of a concrete task to ground some of what we've been talking about here. s121 00:10:43.920 --> 00:10:51.680 I'm going to walk through the database exploration task which is hopefully a little approachable for uh which is hopefully approachable to understand. s122 00:10:51.680 --> 00:11:02.959 If you guys have ever worked in data engineering or know data engineers, you know that one of the things they do is operate over these vast databases with large set many different sets of tables, schemas they're unfamiliar with and idiosyncrasies s123 00:11:02.959 --> 00:11:04.000 in the data. s124 00:11:04.000 --> 00:11:07.360 And if you're a human data engineer, you learn those schemas over time. s125 00:11:07.360 --> 00:11:08.959 You learn how tables link together. s126 00:11:08.959 --> 00:11:11.200 You learn the idiosyncrasies in the data. s127 00:11:11.200 --> 00:11:16.640 But if I was an agent where my memory was being reset every single time, I wouldn't learn any of that, right? s128 00:11:16.640 --> 00:11:31.760 And so here what we can use as a measure for our ability to continually learn is the efficiency in the the task here is you're given a natural language question over the data and you have to answer it by performing some set of SQL queries right that gives you the answer from the from the databases s129 00:11:31.760 --> 00:11:44.640 but if you don't know anything about the databases it's going to take you a lot of SQL queries at the start and at the first questions you're going to be doing a lot of different exploration into figuring out what do the schemas look like what is the data format how do these link together s130 00:11:44.640 --> 00:11:46.800 but as you progress through the benchmark. s131 00:11:46.800 --> 00:11:52.800 If you're a continually learning system, maybe let's say by the 10th instance of a task, it'll look a little different, right? s132 00:11:52.800 --> 00:11:56.480 You'll you'll be able to know a lot more about the schemas, the idiosyncrasies of the data. s133 00:11:56.480 --> 00:12:01.680 You'll know how things link together, and it'll take you much less to actually answer the question. s134 00:12:01.680 --> 00:12:04.800 But there's a bit of a there's a bit of a conundrum here. s135 00:12:04.800 --> 00:12:08.079 We're going to add something a little more challenging to this task, too. s136 00:12:08.079 --> 00:12:11.680 In the real world, there's often concept drift in the things you're doing. s137 00:12:11.680 --> 00:12:14.800 In the database example, let's say there's a migration of your database, right? s138 00:12:14.800 --> 00:12:15.680 Columns get dropped. s139 00:12:15.680 --> 00:12:17.440 There's new columns with different names added. s140 00:12:17.440 --> 00:12:18.800 The data format changes. s141 00:12:18.800 --> 00:12:23.040 As a human, you reason through all this uncertainty and you're able to update your priors. s142 00:12:23.040 --> 00:12:30.880 You know that there's still some information from the past that may be relevant, but from your exploration, you learn what to forget and you learn what's actually relevant to maintain over time. s143 00:12:30.880 --> 00:12:35.920 You kind of have an innate ability to maintain the stability and plasticity trade-off in your mind. s144 00:12:35.920 --> 00:12:38.480 But this isn't native to a lot of language models. s145 00:12:38.480 --> 00:12:47.760 And so if we give it a task after the database migration, it might struggle to require it might struggle in its ability to detect and discard stale experience and simultaneously s146 00:12:47.760 --> 00:12:49.519 update from new experience. s147 00:12:49.519 --> 00:13:00.959 And so this n this notion of concept drifts is something we try to add a lot to our tasks to further test the limits of what memory and continual learning might look like in language model systems. s148 00:13:01.760 --> 00:13:05.680 In continual learning bench 1.0, know which is the benchmark I'm talking about today. s149 00:13:05.680 --> 00:13:08.639 We have tasks from six different domains of tasks. s150 00:13:08.639 --> 00:13:27.040 Okay, we have blind spectrum monitoring which is a signal processing task, codebase adaptation which is a software engineering efficiency, cohort studies and epidemiology task, exploitable poker, a strategic game playing task and because my friends like to play poker, and a database exploration task, the one I just showed you, and sales prediction, which is more of a data s151 00:13:27.040 --> 00:13:29.360 sciency prediction error style task. s152 00:13:29.360 --> 00:13:41.040 Across each of these task domains, we construct sequences of individual task instances with defined reward metrics that agents or models are tasked with solving gradually over time. s153 00:13:41.040 --> 00:13:46.560 And all of these task instances across domains we validate with domain experts to see is this learnable? s154 00:13:46.560 --> 00:13:47.519 Is this realistic? s155 00:13:47.519 --> 00:13:52.639 Is these the sorts of drifts you would expect and things you would expect to remember in an environment. s156 00:13:52.639 --> 00:13:55.360 Let's jump quickly into the results. s157 00:13:55.360 --> 00:14:04.560 In the first in initial release of the continual learning bench, we evaluated a lot of these context management systems to begin with and there's some parametric systems down the line too. s158 00:14:04.560 --> 00:14:17.920 If we look at just aggregate reward for a second and ignore gain and cost, you'll kind of see that the incontext learning systems, this is vanilla in context learning where you just put the experience in the context and you don't do any of the fancy context management that some of these other systems do. s159 00:14:17.920 --> 00:14:21.040 It tops the leaderboard and and it's not just on reward. s160 00:14:21.040 --> 00:14:28.959 It's actually also when we look at the Pareto frontiers, this kind of holds across reward uh versus cost and gain versus cost. s161 00:14:28.959 --> 00:14:32.560 I don't necessarily think this is what the end state of continual learning might look like. s162 00:14:32.560 --> 00:14:42.639 You could argue that these were medium horizon tasks and they didn't push the frontiers of the incontext learning systems enough and I I would say that's fair and that's one of the things on our road map to push those even further. s163 00:14:42.639 --> 00:14:52.720 But it was still surprising that these more expensive context management systems perform a lot poorly compared to just vanilla in context learning on these sets of tasks where you have to do real learning. s164 00:14:52.720 --> 00:14:56.880 These are real tasks that you might expect a normal person to do. s165 00:14:56.880 --> 00:15:04.320 I think it's also interesting to chat briefly about what some of the failure modes look like when we observed in this initial version of the benchmark. s166 00:15:04.320 --> 00:15:11.120 I'm going to argue that most failure modes in continual learning fall on one side of the stability plasticity trade-off. s167 00:15:11.120 --> 00:15:11.279 Okay. s168 00:15:11.279 --> 00:15:11.440 Okay. s169 00:15:11.440 --> 00:15:16.959 And stability is your ability to retain new information in a stable way and use that for future tasks. s170 00:15:16.959 --> 00:15:20.639 While plasticity is your ability to actually learn from new information. s171 00:15:20.639 --> 00:15:27.199 We see usually that with most continual learning methods, any sort of failure mode comes from the inability to do one of these things. s172 00:15:27.199 --> 00:15:29.199 And I'll dive into examples from both. s173 00:15:29.199 --> 00:15:37.600 You don't have to read too much of the text here, but this is our sales prediction task where the model's tasked with making some predictions or forecasting for sales over the course of I think five years. s174 00:15:37.600 --> 00:15:43.199 The model starts by getting feedback that it overpredicted for the last uh the last submission it set. s175 00:15:43.199 --> 00:15:49.120 And so what it does is that it then leads to it revises its prediction downwards to a lower prediction for the set of five years. s176 00:15:49.120 --> 00:15:54.800 And then it gets feedback that actually its underprediction was too much of an underprediction. s177 00:15:54.800 --> 00:15:58.560 And so as a human might naturally do you would go for the middle. s178 00:15:58.560 --> 00:15:59.759 But that's not what the model does. s179 00:15:59.759 --> 00:16:05.519 The model kind of forgot that it had the over prediction to begin with at the start and it just re jumps back right to the over prediction. s180 00:16:05.519 --> 00:16:12.480 This is kind of example of what stability failures look like where it's not retaining past information in improving its future experience. s181 00:16:12.480 --> 00:16:14.639 Plasticity is a bit of a different story. s182 00:16:14.639 --> 00:16:16.399 This is from our epidemiology task. s183 00:16:16.399 --> 00:16:20.720 And the only words I really want you guys to read this is a this is a system with with a notepad. s184 00:16:20.720 --> 00:16:20.880 Okay? s185 00:16:20.880 --> 00:16:27.440 It's a context management system that's given a notepad to externalize its memory to the highlighted text is is the important thing to read here. s186 00:16:27.440 --> 00:16:33.360 Quote, "These seem to be cohort definitions from a different study schema that doesn't apply here. s187 00:16:33.360 --> 00:16:41.600 the study schema did in fact apply here that the the model was unable to update its priors and data from new information and meaningfully improve over time. s188 00:16:41.600 --> 00:16:47.839 It just didn't even recognize that this is something that's relevant to the task and use that to update its priors. s189 00:16:48.639 --> 00:16:52.160 Before I finish, I want to offer a little bit of my broader thoughts on continual learning. s190 00:16:52.160 --> 00:16:57.199 I alluded to this before that I have some opinions on what methods for continual learning might actually look like. s191 00:16:57.199 --> 00:17:07.600 I'm personally quite excited about the parametric methods for continual learning that look at alternative architectures, data, and algorithms jointly to optimize what continual learning should look like. s192 00:17:07.600 --> 00:17:19.760 If you look at language models today, the training stack used to look something simple like this, or maybe it never did, but I'm simplifying for for the sake of this pre-training, then we did supervised fine-tuning, and then we do RHF. s193 00:17:19.760 --> 00:17:23.199 But the stack kind of adapted over the course of the last few years. s194 00:17:23.199 --> 00:17:32.000 Now it might look something more like this where you have mid-training, you have RL for different teacher models and you finish everything off with multi multi-teer on policy distillation. s195 00:17:32.000 --> 00:17:39.280 A lot of the work in continual learning today is how do we take these models that are already trained and then kind of that frozen checkpoint. s196 00:17:39.280 --> 00:17:43.679 How do we figure out methods for continual learning that work after the fact? s197 00:17:43.679 --> 00:17:47.280 But these models were never designed to be continual learners to begin with. s198 00:17:47.280 --> 00:17:56.880 And one of my hypotheses here at least is that we're operating in a bit of a sunk cost fallacy that because we've trained models the way we are today, we need continual learning methods that work on top of that. s199 00:17:56.880 --> 00:18:04.320 If you were to think of designing what continual learning looks like as a first order principle and a first order requirement, it might look nothing like this. s200 00:18:04.320 --> 00:18:11.440 In the purest sense, continual learning might just be one set, one phase of training for continual learning and everything after that is deployment. s201 00:18:11.440 --> 00:18:13.679 You're just the model is interacting in the environment. s202 00:18:13.679 --> 00:18:15.039 It's updating its weights. s203 00:18:15.039 --> 00:18:24.960 there's only one phase of learning and we don't necessarily have to assume that everything is built on top of the way we do language modeling today just because we've spent so much doing that s204 00:18:24.960 --> 00:18:40.720 there there are there are different ways to explore where if you think about continual learning as a first order requirement for your systems more broadly here for a second I think that continual learning is just one example of my group's broader effort to challenge the way we do things in training today s205 00:18:40.720 --> 00:18:46.640 I think that there are larger questions going on though and opportunities to challenge how we do AI research as a whole. s206 00:18:46.640 --> 00:18:55.679 There's a lot of questions around open science, consolidation of power, safety, what the future of things like academia or these third party institutions might look like. s207 00:18:55.679 --> 00:18:59.200 And I think that there's a huge opportunity for us to rethink a lot of that. s208 00:18:59.200 --> 00:19:03.679 And I've spent some time doing some more writing about that that I'd encourage people to check out. s209 00:19:03.679 --> 00:19:14.799 But the crux being there there are opportunities to reimagine what third party institutions for AI research look like and what the future of open science should look like that I think we all need to spend more time getting involved in. s210 00:19:14.799 --> 00:19:18.720 To wrap up, uh, I want to thank a lot of my contributors on continual learning bench. s211 00:19:18.720 --> 00:19:22.480 This is a collaboration across Berkeley, Snorkel, Udub, Madison. s212 00:19:22.480 --> 00:19:30.400 Uh, thanking the snorkel AI for their open their grant through the open benchmarks grant program and the LOD institute via their LOD slingshots program. s213 00:19:30.400 --> 00:19:35.679 Uh, in the road map, we have more domain specific and longer horizon tasks on the benchmark. s214 00:19:35.679 --> 00:19:38.880 We want to add more of the OSS models and the parametric approaches. s215 00:19:38.880 --> 00:19:44.720 We want to improve what simulation for task environments outside of easily verifiable deterministic domains look like. s216 00:19:44.720 --> 00:19:48.000 What would continual learning where you're personalizing to a user model do? s217 00:19:48.000 --> 00:19:53.600 I don't think we have models that are capable enough of simulating that right now to figure it out, but I think we should. s218 00:19:53.600 --> 00:19:57.280 If there's one thing to take away, continual learning doesn't look like point capabilities. s219 00:19:57.280 --> 00:20:03.600 We need to measure it the right way to optimize for the right objective as a field because that's a history of how machine learning has progressed. s220 00:20:03.600 --> 00:20:05.039 Thank you guys very much for listening. s221 00:20:05.039 --> 00:20:07.200 Please feel free to email me or follow for updates. s222 00:20:07.200 --> 00:20:11.360 I would love to stick around, but I'm going to catch a flight, but I'm happy to take anything afterwards via email. s223 00:20:11.360 --> 00:20:14.000 Thank you guys very much.