WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=1P1hJ36rxM0

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/1P1hJ36rxM0.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:13.519 --> 00:00:17.685
Please welcome to the stage the vice president of research

s3
00:00:17.685 --> 00:00:18.160
[music]

s4
00:00:18.160 --> 00:00:22.720
at Google DeepMind, Benois Schillings.

s5
00:00:22.925 --> 00:00:24.925
[music]

s6
00:00:30.785 --> 00:00:32.785
[music]

s7
00:00:49.039 --> 00:00:50.399
All right, good morning.

s8
00:00:50.399 --> 00:00:55.440
Uh this is really quite exciting to be here and have a chance to to speak with all of you.

s9
00:00:55.440 --> 00:00:57.280
Uh my name is Benois Shellings.

s10
00:00:57.280 --> 00:01:02.559
I'm actually a bit of a noob when it comes to to machine learning.

s11
00:01:02.559 --> 00:01:08.560
Till a year and a half ago, I was working for Google X which some of you may know.

s12
00:01:08.560 --> 00:01:13.280
We've done things like Whimo, which seems to be at every street corner now.

s13
00:01:13.280 --> 00:01:15.280
Uh we also do things like Glass.

s14
00:01:15.280 --> 00:01:19.520
So, you know, we we had a mix of hit and success.

s15
00:01:19.520 --> 00:01:27.759
But in many ways this was for me an interesting formative experience on how to run a research team in a place like deep mind.

s16
00:01:27.759 --> 00:01:29.600
I do have an incredible team.

s17
00:01:29.600 --> 00:01:42.720
Uh my team goal in deep mind is basically to develop whatever technology will be needed to make Gemini incredible between one months and one year from now.

s18
00:01:42.720 --> 00:01:54.479
So one month because if you start to work on what is needed in one week that's a very different type of job and one year because I don't think anybody can really predict anything that far.

s19
00:01:54.479 --> 00:02:02.000
So that's already pretty ambitious in my opinion to think about things that would happen one year in the future.

s20
00:02:02.000 --> 00:02:05.920
We do many things under that role.

s21
00:02:05.920 --> 00:02:11.360
Uh a lot of it is related to code which will be the main subject of my talk today.

s22
00:02:11.360 --> 00:02:24.160
uh but we also do a lot of research on what is the evolution of reasoning for models for instance or we do topology research what are new type of network that might bring better performance

s23
00:02:24.160 --> 00:02:38.959
uh we do fundamental work in the science of reinforcement learning which is so fundamental to what we're doing today with ML let's do a bit of an origin story Um,

s24
00:02:40.959 --> 00:02:54.239
we started the project at X named Pitchfork uh in 2018 which was aimed at looking at how ML could really improve the way code is being written.

s25
00:02:54.239 --> 00:03:03.200
And this was very interesting because in 2018 when we presented that at Google honestly nobody would give us the time of day.

s26
00:03:03.200 --> 00:03:09.200
uh there was that point like why would you ever need ML to to write code.

s27
00:03:09.200 --> 00:03:14.319
Um at the same time I think that we totally underestimated how fast this could go.

s28
00:03:14.319 --> 00:03:22.560
When we did that project originally the idea was to look at how we could speed up the evolution of a piece of code.

s29
00:03:22.560 --> 00:03:27.920
How could we make many of those small changes which slows down code speed development?

s30
00:03:27.920 --> 00:03:35.040
you know the small edit which requires a review that takes three days and how we could compress that cycle.

s31
00:03:35.040 --> 00:03:41.519
Some people were talking about vibe coding writing code in English and at the time honestly I totally dismissed that.

s32
00:03:41.519 --> 00:03:43.920
I was that's why we have programming language.

s33
00:03:43.920 --> 00:03:45.920
English is not a programming language.

s34
00:03:45.920 --> 00:03:50.319
Well I I guess I was pretty wrong on that front.

s35
00:03:50.319 --> 00:03:58.319
But the resistance we felt at the time reminded me of how my own career was pretty resistive to to change.

s36
00:03:58.319 --> 00:04:03.360
Um I've been writing code for 45 years.

s37
00:04:03.360 --> 00:04:07.920
Uh I started by writing video game for Apple 2 and Commodore 64.

s38
00:04:07.920 --> 00:04:12.159
So uh my formation was to write assembly language.

s39
00:04:12.159 --> 00:04:18.799
And when you spend a long time writing assembly language, you look at compilers with a lot of suspicion, right?

s40
00:04:18.799 --> 00:04:21.120
are those things really working correctly?

s41
00:04:21.120 --> 00:04:29.520
And then when you switch to C++ and use compiler, you lose you look at garbage collected languages as this h that's not real programming.

s42
00:04:29.520 --> 00:04:31.680
You need to manage your memory.

s43
00:04:31.680 --> 00:04:34.240
Well, today I use Python and V coding.

s44
00:04:34.240 --> 00:04:37.120
So even old dogs can learn new tricks.

s45
00:04:37.120 --> 00:04:42.400
So uh but I I I do understand what happened there.

s46
00:04:43.040 --> 00:04:53.520
I think that we have a number of eras in what happened with software and and the first one was you know the one where I started writing code where the fundamental

s47
00:04:53.520 --> 00:05:05.680
limit was really the machine and there was a lot of work to go and extract the last ounce of power out of those machine and that was the days of

s48
00:05:05.680 --> 00:05:17.440
assembly language where you really needed to be incredibly accurate in the way you were writing code computing became much cheaper and we switched to the modern cloud era where

s49
00:05:17.440 --> 00:05:20.960
getting the best performance is not the most critical aspect.

s50
00:05:20.960 --> 00:05:32.080
You can actually brute force many problems but really what became the limiting factor was the ability for us to design in a modular way.

s51
00:05:32.080 --> 00:05:40.639
You know this was the era where software was write it only once and this was this whole idea of how are you going to build libraries?

s52
00:05:40.639 --> 00:05:42.720
How are you going to write functions?

s53
00:05:42.720 --> 00:05:47.759
How are you going to break down that problem into something that is long-term manageable?

s54
00:05:47.759 --> 00:05:55.280
The limitation there and that determine a lot of how our software process are working where actually the human brain.

s55
00:05:55.280 --> 00:06:03.280
Uh a traditional human typical human is able to get the context between seven and nine tokens.

s56
00:06:03.280 --> 00:06:11.840
I mean we have very rich tokens but you compare that to modern ML where the context is basically going to be infinite pretty soon.

s57
00:06:11.840 --> 00:06:18.240
uh that fundamental limitation of human determined a lot of how software was being written.

s58
00:06:18.240 --> 00:06:26.400
This is over and we're switching now to that AI frontier where really writing the code is not the challenge anymore.

s59
00:06:26.400 --> 00:06:28.080
Uh I'll speak some more about it.

s60
00:06:28.080 --> 00:06:41.440
But the bottlenecks are really how do you ensure that that code is what you really wanted because writing the code is easy but getting what is needed for a specific problem can be much harder to to specify.

s61
00:06:41.440 --> 00:06:49.919
So humans at least in the near future will be that role of architecture or thinking of what are really the implication of that piece of code.

s62
00:06:49.919 --> 00:06:53.120
I'm getting the ML to to design.

s63
00:06:53.120 --> 00:07:08.240
Inductive thinking is another category where I think humans still have a very clear edge which is to look at a system in a much wider context and to be able to detect patterns and from those pattern take some decision.

s64
00:07:08.880 --> 00:07:10.319
So where are we today?

s65
00:07:10.319 --> 00:07:14.960
Um, superhuman syntax generation.

s66
00:07:15.039 --> 00:07:19.280
When is the last time I got Gemini to write a function for me?

s67
00:07:19.280 --> 00:07:23.680
And I looked at the function and I was like, I can do that better.

s68
00:07:23.680 --> 00:07:24.560
It's over.

s69
00:07:24.560 --> 00:07:33.919
Uh, I think that the minutia of code writing, I mean, you can fight, you can argue, you can find counter example, but that time is is gone.

s70
00:07:33.919 --> 00:07:38.400
Where we still have a lot of work to do is multi-step code base.

s71
00:07:38.400 --> 00:07:42.479
Uh software engineering is not about writing code.

s72
00:07:42.479 --> 00:07:53.120
Software engineering is the first time you join a company and you realize that there are 35 million lines of PHP in the codebase and that you need to make some changes.

s73
00:07:53.120 --> 00:08:00.960
That that's the day you understand what software engineering is and that's a place where our models today or frontier models are progressing.

s74
00:08:00.960 --> 00:08:11.280
But this ability to manage that extreme complexity and break it down into man manageable pieces is a place where the frontier is still moving.

s75
00:08:11.280 --> 00:08:15.440
Um it goes all the way to architecture.

s76
00:08:15.440 --> 00:08:19.280
You look at I don't know the Google architecture.

s77
00:08:19.280 --> 00:08:23.919
Thanks God we have Jeff Dean which was you know the the key architect there.

s78
00:08:23.919 --> 00:08:30.240
But that's the level of thinking which has many implication which can go from how do you do hardware optimization?

s79
00:08:30.240 --> 00:08:32.159
How do you manage security?

s80
00:08:32.159 --> 00:08:36.479
How do you build a system so that 10 years later you're not full of regrets?

s81
00:08:36.479 --> 00:08:41.919
And I think this is really the the range of progress we are working on today.

s82
00:08:41.919 --> 00:08:44.640
So code is over but there's plenty to do.

s83
00:08:44.640 --> 00:08:48.000
There's plenty of progress to be made.

s84
00:08:48.080 --> 00:08:55.440
Now code is a very unique problem and in some way that's the reason we we did pitchfork on this.

s85
00:08:55.440 --> 00:09:00.399
Um, first of all, code is a lot of data.

s86
00:09:00.399 --> 00:09:05.519
There are other domains where you can find a lot of data to train your model, but code was so incredible.

s87
00:09:05.519 --> 00:09:09.760
You could go and go on GitHub and start to to scrape GitHub.

s88
00:09:09.760 --> 00:09:16.480
So, this was one of those problem where the amount of training data was a very unique situation.

s89
00:09:16.480 --> 00:09:20.880
It is also a domain where doing verification is reasonable.

s90
00:09:20.880 --> 00:09:24.880
You can run a piece of code, you can compile it, you can have unit test.

s91
00:09:24.880 --> 00:09:33.200
So the ability to figure out is the model generating something correct was something that was pretty reasonable to do.

s92
00:09:33.200 --> 00:09:35.279
That brought us where we are today.

s93
00:09:35.279 --> 00:09:39.519
But today what happened is that we run out of training data.

s94
00:09:39.519 --> 00:09:45.360
I think that 80% of the new code added to GitHub today is machine generated.

s95
00:09:45.360 --> 00:09:53.920
So the notion of human bringing some knowledge that can be used for mining and to train model is reaching an end.

s96
00:09:53.920 --> 00:10:01.040
But the good news is that we can do selfplay and selfplay is something we always liked a lot at deep mind.

s97
00:10:01.040 --> 00:10:03.440
I suppose all know Alpha Zero.

s98
00:10:03.440 --> 00:10:12.640
Alpha Zero became a superhuman go and chess player without any human knowledge just by playing against itself.

s99
00:10:12.640 --> 00:10:20.240
We are now at that stage where Frontier model for code are able to do the same where they can create their own challenge.

s100
00:10:20.240 --> 00:10:22.880
They can judge the validity of the answer.

s101
00:10:22.880 --> 00:10:26.720
they can even to some extent judge the architecture.

s102
00:10:26.720 --> 00:10:36.000
So that ability to do those hundreds of millions of hours of selfplay writing code is the thing that will bring us to the to the next layer.

s103
00:10:36.000 --> 00:10:37.040
You know, it's interesting.

s104
00:10:37.040 --> 00:10:39.600
Um do the experiment.

s105
00:10:39.600 --> 00:10:51.040
Take a a brilliant software engineer, lock him in a room, lock him or her in a room for two years and feed pizza and give the mission you need to become a better software engineer.

s106
00:10:51.040 --> 00:10:52.320
What do you do as a person?

s107
00:10:52.320 --> 00:10:59.279
you you give yourself some challenges, challenges that you can verify and you keep working and coding on those challenges.

s108
00:10:59.279 --> 00:11:00.800
We can do the same here.

s109
00:11:00.800 --> 00:11:05.839
So this is an issue of how much compute, how much selfplay time we can have.

s110
00:11:05.839 --> 00:11:12.160
But that will bring the horizon of how far we go in superhuman coding.

s111
00:11:13.519 --> 00:11:16.800
So the economics of code are changing dramatically.

s112
00:11:16.800 --> 00:11:26.880
You know, as I say, we developed a whole software engineering culture and infrastructure and set of companies based on the assumption that writing code was the hard part.

s113
00:11:26.880 --> 00:11:29.200
That this was the expensive part.

s114
00:11:29.200 --> 00:11:33.839
We're now in a world where writing code is free or nearly free.

s115
00:11:33.839 --> 00:11:37.200
That's why I've got the tilda there.

s116
00:11:37.600 --> 00:11:43.120
That means that the amount of code that we're going to see produced is going to explode.

s117
00:11:43.120 --> 00:11:46.079
And there are some hard implications to that.

s118
00:11:46.079 --> 00:11:49.440
First is the question of design and adequacy.

s119
00:11:49.440 --> 00:11:59.279
How in front of that mountain of code which will be written or written dynamically, how do we keep systems which works and are reliable at the microscopic level?

s120
00:11:59.279 --> 00:12:01.760
Great role for human.

s121
00:12:01.760 --> 00:12:08.880
It is also the issue that you know we're writing code and we're not reading it very much anymore.

s122
00:12:08.880 --> 00:12:20.000
I mean I know we still have code review but I would predict that in one year we'll let Gemini or other model generate the code and nobody will actually look at it.

s123
00:12:20.000 --> 00:12:30.880
You know it's similar to compilers who still check the assembly output of their compiler and maybe someone there but that's probably the end of it.

s124
00:12:30.880 --> 00:12:40.000
So the same thing is going to happen to code and that brings some question of what are the new process that we need to put in place to keep that manageable

s125
00:12:40.000 --> 00:12:45.760
and that's where I've got a [clears throat] a bit of a list active guard rails.

s126
00:12:45.760 --> 00:12:57.120
I mean you've all seen the news of mythos looking at a piece of code and detecting a unreasonable number of vulnerability in that code.

s127
00:12:57.120 --> 00:13:07.440
there is a rush to go and patch those vulnerability but I think that's going to be a neverending process you know we're going to get a certain layer of vulnerability

s128
00:13:07.440 --> 00:13:23.279
discovered by models we're going to fix those models will get smarter they will go a bit deeper and find even more subtle vulnerability so I think that the first aspect is that we need to think at least as much about code security

s129
00:13:23.279 --> 00:13:34.639
and the implication of a piece of code than on the code writing itself and the grail and you know something my team is working actively on is instead of detecting

s130
00:13:34.639 --> 00:13:48.560
the vulnerability and then suggesting some fix how about teaching model to write correct things from the start and that is very very hard to do because it is very context dependent

s131
00:13:48.560 --> 00:14:01.519
the other aspect is that you know that's what I call inductive architecture uh I think that models today are still not very good at transferring knowledge of taking knowledge from one domain

s132
00:14:01.519 --> 00:14:12.480
and applying it to another one or taking two concepts and finding the intersection of those context to be those context to be able to do deductive thinking.

s133
00:14:12.480 --> 00:14:22.240
If we really want to write those very complex software system using ML that is a skill that we need to teach and you know one aspect of that is to really teach models

s134
00:14:22.240 --> 00:14:25.040
how to do correct planning in front of a problem.

s135
00:14:25.040 --> 00:14:36.399
How do you look at a very complex problem and decide what is the right decomposition of that problem that will bring the best clarity or correctness to the to the problem.

s136
00:14:36.399 --> 00:14:39.680
We also need to change the way we do evaluation.

s137
00:14:39.680 --> 00:14:51.199
I mean u threebench is infamous in in my book because threebench verifies if a piece of code runs and produce the right output.

s138
00:14:51.199 --> 00:14:56.160
That's only a small part of as I mentioned earlier of code engineering.

s139
00:14:56.160 --> 00:15:04.399
So for instance, I think that we need some problems much more in those benchmarks that we use which are open-ended problem.

s140
00:15:04.399 --> 00:15:05.920
I I'll give an example.

s141
00:15:05.920 --> 00:15:09.519
Uh I love the question of text compression.

s142
00:15:09.519 --> 00:15:13.199
How many bits per character do you need and how far can you go?

s143
00:15:13.199 --> 00:15:15.680
So that's a very simple eval to to write.

s144
00:15:15.680 --> 00:15:29.360
You just take a piece of 10 megabyte of code and you tell the model write the best compressor you can that is lossless and the loss function in that case will be you know the size of the compressed file plus the size of the source code

s145
00:15:29.360 --> 00:15:42.959
that's never ending I mean those problems are I think what's going to force those model to do novel things like creating totally new algorithmic for instance and I I think we're now getting to that stage

s146
00:15:45.199 --> 00:15:51.920
Writing code or doing software engineering is not thinking as a chain of tokens.

s147
00:15:51.920 --> 00:15:58.880
Thinking and reasoning today is chain of code which has been you know very successful and improve models a lot.

s148
00:15:58.880 --> 00:16:03.839
But humans of course are much more complex in the way they think about problems.

s149
00:16:03.839 --> 00:16:14.480
I always think that code writing is a very visual activity and that can be I don't know the block diagram of what you're doing or the flow of data through your code.

s150
00:16:14.480 --> 00:16:24.079
uh but saying that code will be just a set of token that you emit that are going to be the code I think goes only up to a certain point

s151
00:16:24.079 --> 00:16:38.240
that's a very interesting aspect to what we do at Google Gemini we made that choice from the onset that this would be a multimodal model that you know text was only one of the modality that Gemini would be able to apply

s152
00:16:38.240 --> 00:16:49.759
and we're starting to see you know how can a model start to think in term of spatial or dynamic representation to to solve problem and I think that's going to become

s153
00:16:49.759 --> 00:17:12.959
a must have another interesting question is is this time to create a new language for models Python you name it have been invented for humans and those language are not very good to write safe or reliable code I mean they're great to write code but they're certainly not the the best thing

s154
00:17:12.959 --> 00:17:19.199
I think We're getting to the point where since the pain of writing the code does not exist anymore.

s155
00:17:19.199 --> 00:17:32.160
How about we make writing the code much harder by having you know very strongly typed languages or you know some inspiration from lean on how to write code that by design it's not going to be perfect.

s156
00:17:32.160 --> 00:17:39.679
I mean program proof is something which has some limits but at least putting the burden of correctness on the model.

s157
00:17:39.679 --> 00:17:47.280
So I don't know if we have some language designers here, but I I I think there's something really to be done there and it doesn't need to be human readable.

s158
00:17:47.280 --> 00:17:50.880
I I don't think that that will matter anymore.

s159
00:17:51.280 --> 00:17:57.679
So beyond code, um code is a universal language to solve problems.

s160
00:17:57.679 --> 00:18:10.320
I think that what we're starting to see is this ability to experiment very quickly in code is impacting other domain very quickly because doing experiment becomes basically free.

s161
00:18:10.320 --> 00:18:21.679
So I think that looking at that intersection of code writing and atoms or science is another big front that we are opening that is the place where true novelty

s162
00:18:21.679 --> 00:18:24.400
is going to appear.

s163
00:18:24.880 --> 00:18:29.039
two which are especially exciting for me is chemistry.

s164
00:18:29.039 --> 00:18:36.160
Um you know as humans we do not understand chemistry or we understand a very very small sliver of chemistry.

s165
00:18:36.160 --> 00:18:42.400
Once you have more than 20 atoms in your molecule it's like wow we don't know what that thing is going to do.

s166
00:18:42.400 --> 00:18:45.760
I think we're going to see incredible things emerging out of that.

s167
00:18:45.760 --> 00:18:50.640
I mean once you are able to put 10,000 atom together that starts to look like life.

s168
00:18:50.640 --> 00:18:54.799
So what are all the other things you can do with 10,000 atoms?

s169
00:18:54.799 --> 00:18:55.679
Biology.

s170
00:18:55.679 --> 00:19:06.240
You probably heard plenty about it, but you know, biology is the case of nature did an incredible engineering job and terrible job at documentation.

s171
00:19:06.240 --> 00:19:08.880
Um, but we can crack through that now.

s172
00:19:08.880 --> 00:19:12.799
Models are able to find those relationship that might be elusive for us.

s173
00:19:12.799 --> 00:19:17.600
So I think that that is something that will open incredible door.

s174
00:19:17.600 --> 00:19:21.440
And then there is what I call the gold we cannot see.

s175
00:19:21.440 --> 00:19:25.919
Humans are incredibly biased in what we feel is the correct solution.

s176
00:19:25.919 --> 00:19:31.520
I mean, we're the result of an evolutionary training that help us survive in the jungle, right?

s177
00:19:31.520 --> 00:19:34.000
Not doing quantum computing.

s178
00:19:34.000 --> 00:19:45.440
So, I think that even though we can be brilliant and innovative, there are whole bunch of progress and breakthrough that can be done which we just cannot see or perceive.

s179
00:19:45.440 --> 00:19:47.679
If I had more time, I would give some examples.

s180
00:19:47.679 --> 00:19:59.280
I think that's one of the thing where ML is such a different viewpoint on many of those problems that we're going to get the oh my god this was in front of us the whole time and we could not see it.

s181
00:19:59.280 --> 00:20:01.440
So exciting times ahead.

s182
00:20:01.440 --> 00:20:03.248
Thank you very much.

s183
00:20:03.248 --> 00:20:05.248
[applause]

s184
00:20:23.554 --> 00:20:25.554
[music]