WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=AVMr9PMINyo

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/AVMr9PMINyo.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:12.560 --> 00:00:26.000
This is a discussion that I'm particularly excited about because the field is moving so fast and we have two people that that kind of have this unique vantage point on the field and what I want to do is kind of ask questions to see if we can learn from that.

s2
00:00:26.000 --> 00:00:31.800
So I want to start off with intros, talk a little bit about your role, what you're thinking about, what you're working on.

s3
00:00:31.800 --> 00:00:33.360
Maybe Dan if you can go first.

s4
00:00:33.360 --> 00:00:34.400
Uh hey everyone.

s5
00:00:34.400 --> 00:00:34.840
I'm Dan.

s6
00:00:34.840 --> 00:00:37.600
I'm the VP of Kernels Together AI.

s7
00:00:37.600 --> 00:00:45.400
I lead inference, GPU optimization, trying to figure out how to use GPUs most effectively to serve AI models.

s8
00:00:45.400 --> 00:00:45.680
Yeah.

s9
00:00:45.680 --> 00:00:53.600
So one of the things that I wanted to dive in with Dan about is like you model drops, what are what is everything that goes on behind the scenes

s10
00:00:53.600 --> 00:00:57.280
to serve it so that everybody here there's a lot of builders here that can use it.

s11
00:00:57.280 --> 00:01:01.200
Olive I want to throw it over to you to talk about your role and what you're focusing on.

s12
00:01:01.200 --> 00:01:01.880
Yeah.

s13
00:01:01.880 --> 00:01:09.720
I'm Olive and I am the research lead of RL at MiniMax and I am responsible for the final training of the model and the shipping of the model.

s14
00:01:09.720 --> 00:01:12.240
So basically everything before the inference, right?

s15
00:01:12.240 --> 00:01:13.280
Okay.

s16
00:01:13.280 --> 00:01:13.920
Awesome.

s17
00:01:13.920 --> 00:01:18.320
Um so maybe I wanted to start off this panel is focusing on open source.

s18
00:01:18.320 --> 00:01:23.440
I wanted to start off with this is your strongest model yet, MiniMax M3.

s19
00:01:23.440 --> 00:01:24.560
Why open source it?

s20
00:01:24.560 --> 00:01:29.720
What's the kind of the the idea behind that as a company as you're releasing these models?

s21
00:01:29.720 --> 00:01:35.480
We do believe that the open source community as a whole is very strong and powerful.

s22
00:01:35.480 --> 00:01:38.320
While we open source the model, everyone can use it.

s23
00:01:38.320 --> 00:01:50.600
So it aligns with our mission that we want to have intelligence with everyone and also different developers can contribute to the model through feedbacks, through their own PRs and we can build the models even stronger.

s24
00:01:50.600 --> 00:02:00.680
And also like for example, Dan you you will be able to optimize on our open weight model and make it inference faster and then serve better for everyone.

s25
00:02:00.680 --> 00:02:01.080
Yeah.

s26
00:02:01.080 --> 00:02:08.479
Yeah, we're we're big believers in open source that together and yeah, I think when we we've been you know, following you guys for for a while.

s27
00:02:08.479 --> 00:02:11.680
I think from way old old older MiniMax models.

s28
00:02:11.680 --> 00:02:16.760
So seeing M3 and seeing how far it's come is is really impressive and really great.

s29
00:02:16.760 --> 00:02:20.000
So I wanted to kind of pick on this a little bit more.

s30
00:02:20.000 --> 00:02:23.640
Can you explain so we've got the model creators themselves MiniMax.

s31
00:02:23.640 --> 00:02:26.440
We've got experts on the inference side of things.

s32
00:02:26.440 --> 00:02:27.880
How did this partnership come to be?

s33
00:02:27.880 --> 00:02:31.600
So they launch an open source model and we're now distributing it.

s34
00:02:31.600 --> 00:02:32.560
I I checked this morning.

s35
00:02:32.560 --> 00:02:36.959
We have the lion's share of token usage for MiniMax M3.

s36
00:02:36.959 --> 00:02:41.160
How does this partnership come to be and how do we serve a model like this at scale?

s37
00:02:41.160 --> 00:02:42.080
Yeah, yeah, great question.

s38
00:02:42.080 --> 00:02:49.120
So at together, I think one of the things that that we're really interested in is how do you make intelligence abundant?

s39
00:02:49.120 --> 00:02:58.680
So how do you get more tokens to more people to do more useful things and and and get get all these capabilities into more people's hands.

s40
00:02:58.680 --> 00:03:02.200
So we follow all the open models very closely.

s41
00:03:02.200 --> 00:03:04.680
I I don't remember when exactly we we started part.

s42
00:03:04.680 --> 00:03:06.480
Oh, actually I think I do know this.

s43
00:03:06.480 --> 00:03:10.400
We had a car event in Las Vegas sometime last year.

s44
00:03:10.400 --> 00:03:14.720
And with someone from MiniMax came and there he was like, guys, you really got to serve our next model.

s45
00:03:14.720 --> 00:03:16.840
It's going to be really really great.

s46
00:03:16.840 --> 00:03:18.760
So I think from there we we started talking.

s47
00:03:18.760 --> 00:03:30.360
We were serving MiniMax 2.5 and I think 2.7 for for a while and then when leading up to the launch of of M3, we were we were quite excited about it.

s48
00:03:30.360 --> 00:03:39.600
I think we we're seeing the the usage and what people were doing with it was it was really quite exciting and and so from there that's really where we partner.

s49
00:03:39.600 --> 00:03:41.320
We start working on the model.

s50
00:03:41.320 --> 00:03:49.280
The architecture, optimizing it, figuring out you know, what's the best way to serve inference on it and and all those great pieces.

s51
00:03:49.280 --> 00:03:49.680
Yeah.

s52
00:03:49.680 --> 00:03:52.880
I wanted to actually get into more on the model side of things.

s53
00:03:52.880 --> 00:04:01.760
And so as the creator of a model, as as somebody who's like post-training this thing, the model lands and all the builders that are here start using it.

s54
00:04:01.760 --> 00:04:07.520
From your perspective, um what are the kind of the unique capabilities that you love to see people use it for?

s55
00:04:07.520 --> 00:04:11.560
And what are maybe some of the hidden gems that you haven't You thought, "Oh, people would love to build this."

s56
00:04:11.560 --> 00:04:12.800
but you haven't seen a little bit.

s57
00:04:12.800 --> 00:04:14.120
Could you shed more light on that?

s58
00:04:14.120 --> 00:04:14.480
Mhm.

s59
00:04:14.480 --> 00:04:15.080
Mhm.

s60
00:04:15.080 --> 00:04:27.520
So, MiniMax M3, which was different from the M2 series, was that it was actually multi-modal So, it not only understands text and it not only writes code, it also understands videos and images.

s61
00:04:27.520 --> 00:04:33.760
So, we did see a lot of applications on a gen tech multi-modal agents, which is very cool.

s62
00:04:33.760 --> 00:04:36.840
Um and I would say there are a couple that we can highlight, right?

s63
00:04:36.840 --> 00:04:38.720
For example, computer uses.

s64
00:04:38.720 --> 00:04:47.320
The model can is able to navigate through a computer and then do some pretty good creations with the tools that they they can utilize.

s65
00:04:47.320 --> 00:04:50.560
And also, um you can develop games with the model.

s66
00:04:50.560 --> 00:04:58.200
Um it's very fun that I I don't I think that's one of the hidden gems is that we actually worked on um game development.

s67
00:04:58.200 --> 00:05:01.720
So, the model can help you develop a real cool games.

s68
00:05:01.720 --> 00:05:02.760
Um yeah.

s69
00:05:02.760 --> 00:05:14.160
Yeah, one So, when the when the blog dropped and then the paper dropped, one of the things I noticed was that you guys highlighted SVG bench, you guys highlighted kernel bench, you also touched on OS world.

s70
00:05:14.160 --> 00:05:18.560
Can you talk more about Like, what does it take to post-train, especially for those particular domains?

s71
00:05:18.560 --> 00:05:18.760
Mhm.

s72
00:05:18.760 --> 00:05:19.680
Mhm.

s73
00:05:19.680 --> 00:05:26.080
Uh I would say it's the very important thing is the data and how we define the problems.

s74
00:05:26.080 --> 00:05:28.120
Um it could be very different from different tasks.

s75
00:05:28.120 --> 00:05:30.840
For example, let's say the kernel one, right?

s76
00:05:30.840 --> 00:05:43.480
The It would be very important to design the environments of the data so that we can deliberately train reinforcement learning in those very complex environments and let the model to optimize the kernels themselves

s77
00:05:43.480 --> 00:05:47.120
and iteratively improve the performance.

s78
00:05:47.520 --> 00:05:56.880
And one one aspect that I wanted to talk to you about on the kernel development side of things, um where are you seeing open models when it comes to kernel development?

s79
00:05:56.880 --> 00:05:59.880
You recently released a benchmark specifically for this.

s80
00:05:59.880 --> 00:06:01.560
So, I was wondering if you could talk on that a little bit as

s81
00:06:01.560 --> 00:06:02.960
Yeah, yeah, it's a it's a great question.

s82
00:06:02.960 --> 00:06:11.680
So, I think we're seeing all sorts of models of the closed frontier models and the open models get increasingly better at writing kernels.

s83
00:06:11.680 --> 00:06:18.840
So, we use models all the time when we are developing kernels and and and writing the optimization frameworks.

s84
00:06:18.840 --> 00:06:26.600
I think the the interesting thing that we are starting to look at is this benchmark that that we recently released called um parallel kernel bench.

s85
00:06:26.600 --> 00:06:29.640
So, it actually has a bunch of unsolved problems in it.

s86
00:06:29.640 --> 00:06:35.440
So, we went around surveyed all the different ways they can serve model inference.

s87
00:06:35.440 --> 00:06:43.240
And one of the interesting things that we found is that there's a lot of things that we can think of that would actually speed models up that there don't exist good kernels for.

s88
00:06:43.240 --> 00:06:51.880
So, one of the reasons that we put that benchmark out was, you know, one thing that people worry about is like bench maxing or or overfitting to particular benchmarks.

s89
00:06:51.880 --> 00:06:59.240
One of our intentions with this benchmark was that if you overfit to it, that's great cuz we'll go take those kernels and use them to to to accelerate the

s90
00:06:59.240 --> 00:07:01.560
the the the inference and the development.

s91
00:07:01.560 --> 00:07:02.840
Yeah, this is a really interesting point.

s92
00:07:02.840 --> 00:07:13.280
A lot of people have problems with bench maxing, but the way I think about it is if researchers like you put all the really useful benchmarks out and we bench max on all of them and everything is in distribution,

s93
00:07:13.280 --> 00:07:14.560
then that's a perfect world, right?

s94
00:07:14.560 --> 00:07:17.440
That's a very useful model that we can then use.

s95
00:07:17.440 --> 00:07:18.720
Um Okay, cool.

s96
00:07:18.720 --> 00:07:22.200
So, I wanted to touch on the inference side of things now as well.

s97
00:07:22.200 --> 00:07:29.400
So, like a new model drops like this, what does it take Could you take me like behind the scenes at the inference stack?

s98
00:07:29.400 --> 00:07:34.480
And what does it take to go from day zero launch and then optimizing it week over week, month over month?

s99
00:07:34.480 --> 00:07:35.560
Yeah, yeah, great question.

s100
00:07:35.560 --> 00:07:39.640
So, when we partner with someone like MiniMax, we will get some early model details.

s101
00:07:39.640 --> 00:07:48.720
So, for M3, for example, there are things like the minimax sparse attention and and some of those choices that were a little bit different from any model that's out there.

s102
00:07:48.720 --> 00:07:53.600
And I think if you look at any of the open models now, they are all quite different from each other in different ways.

s103
00:07:53.600 --> 00:07:59.640
So, there's different attention, different MOE choices, differences in quantization and and all these pieces.

s104
00:07:59.640 --> 00:08:07.720
So, as soon as we get those details, we start writing kernels, benchmarking, figuring out is there existing kernels that work for it?

s105
00:08:07.720 --> 00:08:09.520
Do we need to modify something?

s106
00:08:09.520 --> 00:08:12.080
Do we need to write something from scratch?

s107
00:08:12.080 --> 00:08:15.800
And then so day zero, we're trying to think about things like quality.

s108
00:08:15.800 --> 00:08:21.040
So, when this model launches, is it going to have the quality that we all expect?

s109
00:08:21.040 --> 00:08:26.640
Are we going to be able to provide the right the the right user experience?

s110
00:08:26.640 --> 00:08:37.120
And then from there, as soon as it launches that day zero, we have a long list of things that we know, hey, we have to do this with the KV cache, we have to do this with the attention kernels, we have to do

s111
00:08:37.120 --> 00:08:40.520
we're going to look at this part of the quantization, and things like that.

s112
00:08:40.520 --> 00:08:50.400
So, we we we have that list and then we start working on it and start optimizing over the course of weeks so that when you use these models, they actually get faster

s113
00:08:50.400 --> 00:08:55.160
between day zero and day seven and day 14 and etc.

s114
00:08:55.160 --> 00:09:01.480
Yeah, I was just talking to Ingrid actually yesterday and I and I asked her, have we been improving the performance of M3?

s115
00:09:01.480 --> 00:09:04.600
And I meant over the last month and she said, oh, did you mean from last night?

s116
00:09:04.600 --> 00:09:06.640
And this is the pace at which these guys work.

s117
00:09:06.640 --> 00:09:09.200
So, it's very real.

s118
00:09:09.200 --> 00:09:13.320
One aspect that I wanted to touch on with this is we're seeing the workloads shift.

s119
00:09:13.320 --> 00:09:26.000
We're going from kind of predominantly chat workloads where you have turns coming in now to agentic workloads where you've got this thing sitting inside a harness and you're doing hundreds and hundreds of multi-turn tool calls.

s120
00:09:26.000 --> 00:09:26.240
Yeah.

s121
00:09:26.240 --> 00:09:29.640
Does that change the way you build the inference stack?

s122
00:09:29.640 --> 00:09:30.640
Yeah, it definitely does.

s123
00:09:30.640 --> 00:09:40.560
So, these agentic turn-based workloads, they go into everything from informing your KV cash, your prompting, your pieces like this.

s124
00:09:40.560 --> 00:09:50.600
So, it it informs what part of the stack you want to go optimize because now I think when we're in the chat chat chat world, you have a system prompt of a few thousand,

s125
00:09:50.600 --> 00:09:52.160
and then you just have the the chat logs.

s126
00:09:52.160 --> 00:10:03.320
Now, with the coding base agentic workflows, you'll upload your whole code base to the model, and that's a very different optimization and routing and kernel challenge than than just the the chat base workload.

s127
00:10:03.320 --> 00:10:05.800
So, yeah, we're we we follow these work codes very closely.

s128
00:10:05.800 --> 00:10:14.080
It's really interesting to see how they evolve and and how to adapt the inference stack and the inference engines to to really take to really serve them well.

s129
00:10:14.080 --> 00:10:18.520
Not only do you have agentic workloads, but you've also got multimodal workloads in there.

s130
00:10:18.520 --> 00:10:27.440
So, what I like to do often with these coding agents is get them to optimize a web app and then get it to use it and then do a feedback loop.

s131
00:10:27.440 --> 00:10:36.120
So, one thing that I wanted to come to you all the four is Minimax M3 is multimodal, M2.7, all the ones before that were not multimodal.

s132
00:10:36.120 --> 00:10:41.720
And like, can you talk a little bit about the optimizations and how you trained it for that aspect?

s133
00:10:41.720 --> 00:10:44.600
And then also, I want to get into the architecture of it afterwards as well.

s134
00:10:44.600 --> 00:10:45.560
Right.

s135
00:10:45.560 --> 00:10:46.080
Definitely.

s136
00:10:46.080 --> 00:10:51.839
So, what's different from before was that it was trained multimodal from scratch.

s137
00:10:51.839 --> 00:10:56.320
So, from step zero, we trained not only text data, we also trained image data.

s138
00:10:56.320 --> 00:11:05.080
And it was normal for many other labs that the model would collapse after training a little bit, and we managed to solve that problem.

s139
00:11:05.080 --> 00:11:16.680
And what we found was actually that with this kind of training from scratch, if you look at the attention map, it actually the visual of the the text tokens would attend to the visual tokens.

s140
00:11:16.680 --> 00:11:20.120
So, that they are naturally combined together, they naturally understand each other.

s141
00:11:20.120 --> 00:11:25.600
So, for example, we are developing, for example, websites, right?

s142
00:11:25.600 --> 00:11:29.680
It is better if we train with both modalities.

s143
00:11:29.680 --> 00:11:35.920
Also, like, for example, it can look at the website, it can understand how it looks, and then better optimize for it.

s144
00:11:35.920 --> 00:11:38.560
Like for example, during reinforcement learning.

s145
00:11:38.560 --> 00:11:42.240
Um so yeah, I think that is pretty cool.

s146
00:11:42.880 --> 00:11:49.720
So one one thing that kind of stuck out with this model for me was the fact that it introduced a lot of new things.

s147
00:11:49.720 --> 00:11:56.000
The multi-modality, the increase of context to 1 million, um the fact that you have a sparse attention now.

s148
00:11:56.000 --> 00:11:59.720
Um so if you go to the inference side, it's almost a a nightmare, isn't it?

s149
00:11:59.720 --> 00:12:04.160
You get You get this new model and there's so many things that you could optimize to speed up inference.

s150
00:12:04.160 --> 00:12:07.520
Um practically, what are the things that you focus on?

s151
00:12:07.520 --> 00:12:11.760
There's like a thousand things that you could optimize, but where do you get the most bang for your buck?

s152
00:12:11.760 --> 00:12:14.560
I mean, you focus on a thousand and one things, yeah.

s153
00:12:14.560 --> 00:12:16.800
Like you just you just go and you keep you keep doing it.

s154
00:12:16.800 --> 00:12:21.440
You find every edge that you can, um and and you go and and you push on it.

s155
00:12:21.440 --> 00:12:27.720
Um so yeah, I think there there's there's no there's no stone that you leave unturned and you just uh keep keep going at it.

s156
00:12:27.720 --> 00:12:32.600
If someone tells me you can't do the thousand first thing, yeah, like I don't know, try harder.

s157
00:12:32.600 --> 00:12:33.640
Yeah.

s158
00:12:33.640 --> 00:12:35.680
And then get up a little later.

s159
00:12:35.680 --> 00:12:36.960
Yeah, like go ahead.

s160
00:12:36.960 --> 00:12:38.240
No, I know.

s161
00:12:38.240 --> 00:12:43.839
So the other thing that I wanted to ask is there's a whole kind of um zoo of open source models.

s162
00:12:43.839 --> 00:12:51.200
As you're talking about speeding up inference, are there things that are there lessons that you can take from one model and apply it to MiniMax M3?

s163
00:12:51.200 --> 00:12:58.280
Um or or do you have to like restart from scratch as you're thinking about the inference engine, the kernels, like how does that work?

s164
00:12:58.280 --> 00:12:58.920
Right.

s165
00:12:58.920 --> 00:13:03.920
Um yeah, so there there's definitely things that you learn from optimizing one model that you take to another.

s166
00:13:03.920 --> 00:13:08.480
Um so I think sparse attentions are something that have become quite popular now.

s167
00:13:08.480 --> 00:13:18.240
So the MiniMax sparse attention is a little bit different from the uh from the deep seek and the and and and those and the and the ones that you find in JLM.

s168
00:13:18.240 --> 00:13:27.960
Um but the they're still similar lessons that you can take from that optimization process and that kernel writing process that you can then bring to to the new sparse attentions.

s169
00:13:27.960 --> 00:13:35.160
And you know, we've been in in some form or or another I've been thinking about this problem for for many years.

s170
00:13:35.160 --> 00:13:37.160
So going all the way back to my PhD.

s171
00:13:37.160 --> 00:13:46.040
So it's it's it's great to see some validation that that folks can now train it at scale and people are using it and and it's and it's it's going pretty well.

s172
00:13:46.040 --> 00:13:46.800
Yeah.

s173
00:13:46.800 --> 00:13:55.960
Yeah, one of the interesting things especially about open source is you've got all these labs that are learning from each other kind of taking the wins from each other, right?

s174
00:13:55.960 --> 00:14:02.360
So if if if one lab figures out that Minimax does this really well, then that gets becomes the golden standard.

s175
00:14:02.360 --> 00:14:14.480
Um one of the things on model launch that I in the blog post that you guys go into is that this model was actually able to replicate a 12-hour run where it could reproduce an ICLR paper.

s176
00:14:14.480 --> 00:14:18.400
And so somebody who's training this model to do this thing, how do you actually go about that?

s177
00:14:18.400 --> 00:14:21.120
Cuz that seems like a pretty ludicrous task.

s178
00:14:21.120 --> 00:14:21.480
Right.

s179
00:14:21.480 --> 00:14:35.080
So letting the model to do cool stuff like replicating papers, optimizing kernel frameworks and stuff like that is always exciting for us researchers because it's like very related to our job.

s180
00:14:35.080 --> 00:14:44.240
But training it can be very tricky because it's very long horizon and like the task itself would require GPUs.

s181
00:14:44.240 --> 00:14:46.560
It has hardware constraints.

s182
00:14:46.560 --> 00:14:59.040
So it's very interesting to train tasks like that and I would say the key there is still the environments and the data and how you formulate the problem, how you formulate the rewards, how you formulate the environment

s183
00:14:59.040 --> 00:15:11.960
and how you change the reinforcement learning algorithm a little bit so that it's trained more efficiently so that you can see cool things emerging through the the iterations of RL runs.

s184
00:15:11.960 --> 00:15:13.040
Mhm.

s185
00:15:13.040 --> 00:15:15.560
Can you talk maybe if I keep pulling on this thread a little bit?

s186
00:15:15.560 --> 00:15:19.640
How do you do evaluation over these longer longer scale runs?

s187
00:15:19.640 --> 00:15:26.320
So if you're if you want the thing to do a 12-hour task, yes, it might or might not do it at the end, but are there like a intermediate

s188
00:15:26.320 --> 00:15:28.040
things that you can also look at?

s189
00:15:28.040 --> 00:15:29.280
Yes, we do.

s190
00:15:29.280 --> 00:15:32.720
For these tasks, they are there are iterations, right?

s191
00:15:32.720 --> 00:15:37.040
So, the model can submit several times, and we would evaluate each of them.

s192
00:15:37.040 --> 00:15:48.080
Some of them some of the times the models would hack, and we do do like validation and test with for it to test if it's really improving on the performance or it's hacking.

s193
00:15:48.080 --> 00:15:51.520
And also, we design our internal evaluations.

s194
00:15:51.520 --> 00:15:57.920
So, for example, for the release of 2.7, we touched a bit on self-evolution, right?

s195
00:15:57.920 --> 00:16:11.800
So, we're actively using the model to improve the speed of development internally, which like out of it out of it we can build our own evaluations that are closely related to our own work that we can evaluate the models on.

s196
00:16:11.800 --> 00:16:12.840
Yeah.

s197
00:16:12.840 --> 00:16:22.520
When So, when we start talking about these long horizon tasks that are 12 hours long, we gave an entire workshop on this on Monday, but what I wanted to come to you, Dan, for is

s198
00:16:22.520 --> 00:16:23.000
KB cache.

s199
00:16:23.000 --> 00:16:33.600
So, if let's say you you have concurrent requests that are 500 to a million thousand context length long, how do you deal with the KB cache that just keeps on growing, and how does the infrastructure deal with that?

s200
00:16:33.600 --> 00:16:37.360
Yeah, so there there's a lot of different pieces that that you put there.

s201
00:16:37.360 --> 00:16:41.680
Like in in some sense, it's like recreating a distributed file system.

s202
00:16:41.680 --> 00:16:47.600
So, we we're in some sense building something like that or a very big database.

s203
00:16:47.600 --> 00:16:49.440
It's it's pretty simple in theory.

s204
00:16:49.440 --> 00:16:59.080
It's like the type of thing that you should have done in your third year of undergrad or something like that, but most of us actually skipped that class, so now we're rediscovering it live in in industry.

s205
00:16:59.080 --> 00:17:02.680
But it's it's all about where do you store that cache?

s206
00:17:02.680 --> 00:17:05.000
How do you know Have you have you seen this before?

s207
00:17:05.000 --> 00:17:05.680
How do you fetch it?

s208
00:17:05.680 --> 00:17:08.760
How do how do you send it from one place to another?

s209
00:17:08.760 --> 00:17:15.920
So, yeah, it's it's a it's it's not that complicated, but you you do have to make sure that that you do a good job.

s210
00:17:15.920 --> 00:17:26.400
Yeah, one thing I noticed you you gave a a um a lecture at Stanford recently and one thing that stood out was that if you fast forward like 2 3 years, 2 3 years is is is a long time in AI.

s211
00:17:26.400 --> 00:17:32.160
And if you look back you said that we'll realize that how um how early we are right now.

s212
00:17:32.160 --> 00:17:32.400
Yeah.

s213
00:17:32.400 --> 00:17:38.520
So, from your vantage point, where 3 years out, what do you think we'll look back on be like, "Why were we doing it this way?"

s214
00:17:38.520 --> 00:17:39.480
Great question.

s215
00:17:39.480 --> 00:17:45.760
Um uh some things that I hope for, so I think we underutilize our GPUs a lot right now.

s216
00:17:45.760 --> 00:17:50.240
Um you know, SpaceX said they would have like 10% flop utilization or something like that.

s217
00:17:50.240 --> 00:17:56.000
I hope in 3 years, well, they they should already be embarrassed about it, but I hope in 3 years they're extra embarrassed by it.

s218
00:17:56.000 --> 00:18:00.880
Um so, certainly training should be should be pretty um should be pretty good.

s219
00:18:00.880 --> 00:18:08.160
I think at Anthropic we can do a lot better with the hardware that we're using uh that we have today, so I hope uh in a few years we'll we'll say

s220
00:18:08.160 --> 00:18:12.440
uh we'll we'll have seen the light on on some of those pieces.

s221
00:18:12.440 --> 00:18:15.840
Um and I think there'll be a lot more models, there'll be a lot better.

s222
00:18:15.840 --> 00:18:21.920
Um I I hope finally by then we've put uh put to bed this question about the open models.

s223
00:18:21.920 --> 00:18:27.960
You know, there's every few months there's someone like, "Oh, Anthropic OpenAI, they're so ahead, yada yada."

s224
00:18:27.960 --> 00:18:36.720
Um but I think uh we're we're seeing with models like M3 and GLM and Kimmy and and all those models that um the open-source frontier really can catch up.

s225
00:18:36.720 --> 00:18:41.280
Um and and it's it's not even that far behind, so so I think that that's quite exciting.

s226
00:18:41.280 --> 00:18:42.000
Yeah.

s227
00:18:42.000 --> 00:18:50.440
I I wanted to throw that same question over to you, Olive, but you mentioned that you it for for the M2 series and the M3 series, you're using this idea of self-evolution,

s228
00:18:50.440 --> 00:18:55.200
where the model is building its own harness, and then it's training inside of that, and then you get the next checkpoint.

s229
00:18:55.200 --> 00:19:04.120
Um if you look back 3 years uh out and then you say like, "What in RL or post-training um do you think uh made the biggest difference?"

s230
00:19:04.120 --> 00:19:06.800
What do you think that is from this vantage point?

s231
00:19:06.800 --> 00:19:08.480
Um great question.

s232
00:19:08.480 --> 00:19:11.503
But, 3 years ago I was still in in

s233
00:19:11.503 --> 00:19:12.240
[laughter]

s234
00:19:12.240 --> 00:19:14.600
I actually didn't start this industry yet.

s235
00:19:14.600 --> 00:19:14.960
Yeah.

s236
00:19:14.960 --> 00:19:17.720
So I wouldn't have imagined what's happening right now today.

s237
00:19:17.720 --> 00:19:19.000
So it's really exciting.

s238
00:19:19.000 --> 00:19:28.840
But I can see how models that were developed are were already improving the speed of development maybe a year ago or even further than a year ago.

s239
00:19:28.840 --> 00:19:32.840
So I could see how this speed is actually accelerating.

s240
00:19:32.840 --> 00:19:34.720
How the development is accelerating.

s241
00:19:34.720 --> 00:19:47.720
And that's how like open weight models can really catch up with frontier labs and Yeah, that's how we think we are more mission to bring this model to everyone

s242
00:19:47.720 --> 00:19:49.360
so that everyone can use it.

s243
00:19:49.360 --> 00:19:50.200
Yeah.

s244
00:19:50.200 --> 00:19:50.640
Awesome.

s245
00:19:50.640 --> 00:19:51.160
Thank you guys.

s246
00:19:51.160 --> 00:19:51.680
Thank you Dan.

s247
00:19:51.680 --> 00:19:53.440
Thank you Thank you Olive.

s248
00:19:53.440 --> 00:19:54.560
Thank you guys so much.

s249
00:19:54.560 --> 00:19:55.320
Have a great day.

s250
00:19:55.320 --> 00:19:55.840
Very cool.

s251
00:19:55.840 --> 00:19:58.080
Thanks so much.

s252
00:20:11.108 --> 00:20:13.108
[music]