WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=Ki980nV0__0

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/Ki980nV0__0.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.760 --> 00:00:14.040
Um welcome.

s3
00:00:14.040 --> 00:00:15.800
Let's get started.

s4
00:00:15.800 --> 00:00:19.680
So, my name is Drupad I want to talk to you about an argument.

s5
00:00:19.680 --> 00:00:28.560
There's an argument online that says, roughly speaking, AI agents will be the drivers of actions on the web, not humans.

s6
00:00:28.560 --> 00:00:35.760
That there will be more agents clicking buttons, reading things, buying things for us than human eyeballs on the web.

s7
00:00:35.760 --> 00:00:40.720
Um you drive one step deeper into the argument and you ask, "How will this happen?"

s8
00:00:40.720 --> 00:00:45.400
Because today the web is extremely hostile to automated traffic.

s9
00:00:45.400 --> 00:00:49.480
And the answer usually is, "The web will be agentified."

s10
00:00:49.480 --> 00:00:55.120
Um you drive one step deeper into, "What does that mean and how will that happen?"

s11
00:00:55.120 --> 00:01:01.400
And the answer usually is, "With APIs that agents will call via some set of protocols."

s12
00:01:01.400 --> 00:01:03.840
And the set of protocols grows over time.

s13
00:01:03.840 --> 00:01:11.200
It was initially supposed to be MCP servers, then web MCP, and for payments there are 20 different competing protocols.

s14
00:01:11.200 --> 00:01:15.800
And you know, every single company wants to introduce their own protocol along the way.

s15
00:01:15.800 --> 00:01:19.400
But okay, there will be APIs is the is the argument.

s16
00:01:19.400 --> 00:01:24.360
My claim today and argument and what I hope to convince you today is that this last bit is wrong.

s17
00:01:24.360 --> 00:01:27.200
I think the first two I generally agree with.

s18
00:01:27.200 --> 00:01:34.120
This last bit that suddenly the web will provide you APIs for accessing things is is just delusional.

s19
00:01:34.120 --> 00:01:41.440
And my claim is computer use agents and computer use models will agentify the web, not APIs.

s20
00:01:41.440 --> 00:01:44.240
And more specifically, the long tail of the web.

s21
00:01:44.240 --> 00:01:51.120
Um the head of the distribution, the most popular website, perhaps will give you the API, but the long tail will not.

s22
00:01:51.120 --> 00:01:55.840
Um and the story starts uh usually with these sorts of use cases.

s23
00:01:55.840 --> 00:02:11.160
I I agree that they also frustrate me when people show a use case of find me business class flights from Miami to Palma de Mallorca for August and somehow they expect that behind the scenes it'll be a computer use agent clicking buttons on flights.google.com.

s24
00:02:11.160 --> 00:02:13.959
I find this example, this is my own example.

s25
00:02:13.959 --> 00:02:15.480
I I've not dunked you on everybody else.

s26
00:02:15.480 --> 00:02:17.320
I find this example bizarre.

s27
00:02:17.320 --> 00:02:19.440
Why would you possibly do it this way?

s28
00:02:19.440 --> 00:02:22.920
Aren't you aware that there are already aggregators?

s29
00:02:22.920 --> 00:02:26.560
You send them a variable, they will send you a JSON.

s30
00:02:26.560 --> 00:02:28.600
Your LLMs can do tool calling.

s31
00:02:28.600 --> 00:02:30.000
Why would you click buttons?

s32
00:02:30.000 --> 00:02:34.080
The purpose of those clicking buttons is to generate this behind the scenes.

s33
00:02:34.080 --> 00:02:36.680
So, we're on the same page there.

s34
00:02:36.680 --> 00:02:40.880
The next step usually is, okay, let's roll this out to some other case.

s35
00:02:40.880 --> 00:02:50.760
Um, I want to know, does my favorite restaurant or does this restaurant have gluten-free items on the restaurant's menu?

s36
00:02:50.760 --> 00:02:59.800
And so, I assume that there's going to be a similar endpoint somewhere for that restaurant website or via an aggregator.

s37
00:02:59.800 --> 00:03:01.800
Oops.

s38
00:03:02.840 --> 00:03:11.680
Um, that accepts a query that I can filter through, where I can ask for give me your menu items that are gluten-free.

s39
00:03:11.680 --> 00:03:19.800
Um, I want to for those of you who can already see, I want to rid you of the delusion that such an endpoint exists.

s40
00:03:19.800 --> 00:03:26.800
Uh, and I want to show you just for the sake of, you know, being on the same page, what two restaurant pages look like.

s41
00:03:26.800 --> 00:03:29.680
Um, there are three that are on the screen.

s42
00:03:29.680 --> 00:03:30.560
This is the first one.

s43
00:03:30.560 --> 00:03:33.920
This is what we imagine a prototypical, this is easy mode.

s44
00:03:33.920 --> 00:03:38.239
You go to a web page, there's no API, but at least it's text.

s45
00:03:38.239 --> 00:03:40.360
It's in German, sure.

s46
00:03:40.360 --> 00:03:43.360
Different part of the world, models can speak languages.

s47
00:03:43.360 --> 00:03:46.200
I see text and I see pricings.

s48
00:03:46.200 --> 00:03:47.959
Maybe this is easily scrapable.

s49
00:03:47.959 --> 00:03:48.880
Okay.

s50
00:03:48.880 --> 00:03:50.760
This is easy mode.

s51
00:03:50.760 --> 00:03:59.440
The medium mode is a uh uh uh a page like this where of course the thing that you're looking for is slightly hidden away.

s52
00:03:59.440 --> 00:04:02.280
You press the menu button, it actually takes you to a PDF.

s53
00:04:02.280 --> 00:04:03.640
Okay, fine.

s54
00:04:03.640 --> 00:04:08.520
If your agent has to do this, it needs a PDF reader mode, fine.

s55
00:04:08.520 --> 00:04:10.400
Like, you know, I can highlight these.

s56
00:04:10.400 --> 00:04:12.640
Okay, so this is at least text, no problem.

s57
00:04:12.640 --> 00:04:17.519
Like, you know, I will dump this into ChatGPT or whatever and it'll do it.

s58
00:04:17.519 --> 00:04:19.359
Here's the hard mode.

s59
00:04:19.359 --> 00:04:34.760
This is what a hard mode website for a restaurant looks like where it's just pictures of the people who created the restaurant, um of where they're located, of menu items,

s60
00:04:34.760 --> 00:04:37.280
and you're like, "Where is the menu?

s61
00:04:37.280 --> 00:04:40.000
Uh does anybody speak Spanish?"

s62
00:04:40.000 --> 00:04:42.240
Nuestra carta, see?

s63
00:04:42.240 --> 00:04:45.200
Okay, let's let's check that's our menu.

s64
00:04:45.200 --> 00:04:47.560
Um you click there.

s65
00:04:47.560 --> 00:04:51.080
Oh my god, these are What am I staring at?

s66
00:04:51.080 --> 00:04:52.480
Okay, uh you click on it.

s67
00:04:52.480 --> 00:04:56.840
Oh, it's a gallery of of individual What is this?

s68
00:04:56.840 --> 00:05:02.160
This is a This is a pixelated This is There's no text here.

s69
00:05:02.160 --> 00:05:08.360
I am looking at JPEGs embedded in a gallery which contain the PDF items.

s70
00:05:08.360 --> 00:05:15.720
You download this and you put it into ChatGPT and it's having It's struggling doing OCR on this thing.

s71
00:05:15.720 --> 00:05:19.160
Okay, so this is what the web looks like.

s72
00:05:19.160 --> 00:05:28.360
You're telling me this will give you an API endpoint that you can pass gluten-free items and it'll tell you what that is.

s73
00:05:28.360 --> 00:05:30.840
That's an example number one.

s74
00:05:30.840 --> 00:05:33.480
Let's take a look at another examples.

s75
00:05:33.480 --> 00:05:37.040
Uh this is much more, you know, enterprise-focused.

s76
00:05:37.040 --> 00:05:40.040
Um um I am a business.

s77
00:05:40.040 --> 00:05:42.800
I want to sell to a school district.

s78
00:05:42.800 --> 00:05:45.760
There's about 15 to 20,000 school districts in the US.

s79
00:05:45.760 --> 00:05:56.000
And I want to ask the question, is this school district where I give you, you know, mydistrict.gov, is this school district procuring a laptop right now?

s80
00:05:56.000 --> 00:05:56.240
Right?

s81
00:05:56.240 --> 00:06:00.040
It's a simple question, and I want a similar endpoint.

s82
00:06:00.040 --> 00:06:04.600
I want to say RFP status open and topic laptop.

s83
00:06:04.600 --> 00:06:07.720
And you think this will exist.

s84
00:06:07.720 --> 00:06:11.560
Uh let me tell you what these websites look like again.

s85
00:06:11.560 --> 00:06:15.560
Um this is what easy mode looks like.

s86
00:06:15.560 --> 00:06:18.440
This is, you know, Ithaca public schools.

s87
00:06:18.440 --> 00:06:20.960
Um okay, there's some portal.

s88
00:06:20.960 --> 00:06:26.000
Um maybe there's, you know, some, you know, enrollment is maybe the wrong place to go.

s89
00:06:26.000 --> 00:06:29.800
This is what slightly harder mode looks like.

s90
00:06:29.800 --> 00:06:32.760
Um this is a different public school.

s91
00:06:32.760 --> 00:06:41.000
You look at the menu, under finance, there's purchasing, under purchasing, there's certain solicitations.

s92
00:06:41.000 --> 00:06:42.520
Um scan PDF again.

s93
00:06:42.520 --> 00:06:43.840
Okay, very nice.

s94
00:06:43.840 --> 00:06:49.360
Uh so, no text in here, but this is how they tell you about what they have have purchased.

s95
00:06:49.360 --> 00:07:06.040
Um and for ultimate boss level, I want to show you a different school district where in order to find information, you have to file a request for access of info or information under Freedom of Information Act.

s96
00:07:06.040 --> 00:07:17.320
And then what they do is they scan your email that you sent to them, and they will scan that email, put it on a Google Drive, and then attach PDFs

s97
00:07:17.320 --> 00:07:19.960
associated with your request.

s98
00:07:19.960 --> 00:07:25.120
These are the people you're telling me will give you an MCP server?

s99
00:07:25.360 --> 00:07:29.680
The the amount of delusion here is off the charts.

s100
00:07:29.680 --> 00:07:35.230
Like Like if there was ever a time to to say go touch grass, like

s101
00:07:35.230 --> 00:07:36.240
[laughter]

s102
00:07:36.240 --> 00:07:38.040
I think this is it.

s103
00:07:38.040 --> 00:07:39.040
Okay.

s104
00:07:39.040 --> 00:07:49.240
Uh so, this is what My claim here is that the web we forget is massive.

s105
00:07:49.240 --> 00:07:51.080
It is extremely big.

s106
00:07:51.080 --> 00:07:57.160
The the number of websites out there somewhere, you know, the active websites are somewhere 200 million.

s107
00:07:57.160 --> 00:08:00.120
The total number of websites are sitting in a in a billion.

s108
00:08:00.120 --> 00:08:03.600
Infrastructure changes very slowly.

s109
00:08:03.600 --> 00:08:16.440
Um you can imagine as an engineer you know, getting unfettered access to software systems and letting your favorite coding agent rip and generate an API endpoint.

s110
00:08:16.440 --> 00:08:25.760
Even if that technological problem is solved, which I agree, seems like on a horizon it should eventually be solved, you're not going to get unfettered access to these institutions

s111
00:08:25.760 --> 00:08:28.800
and these institutions change very slowly.

s112
00:08:28.800 --> 00:08:31.680
There are still places that are faxing each other.

s113
00:08:31.680 --> 00:08:35.760
You're you're not going to be able to suddenly change this overnight.

s114
00:08:35.760 --> 00:08:36.719
Okay.

s115
00:08:36.719 --> 00:08:39.360
So at this point of time you might be thinking, "Fine.

s116
00:08:39.360 --> 00:08:41.280
I'm stuck with infra as it is.

s117
00:08:41.280 --> 00:08:44.520
Um so there's not going to be APIs available.

s118
00:08:44.520 --> 00:08:45.680
But I have coding agents.

s119
00:08:45.680 --> 00:08:48.440
Why don't I just throw them at the HTML?

s120
00:08:48.440 --> 00:08:52.440
There is after all, if the browser's doing it, it's a piece of code.

s121
00:08:52.440 --> 00:08:55.280
My coding agent should be able to do it as well."

s122
00:08:55.280 --> 00:08:56.640
Fair fair point.

s123
00:08:56.640 --> 00:08:59.480
I you know, I also thought this way two years ago.

s124
00:08:59.480 --> 00:09:03.040
Let me tell you what what the web actually looks like.

s125
00:09:03.040 --> 00:09:11.120
Um If I ask you the question, what was the final score of this game between Minnesota Timberwolves and the Brooklyn Nets?

s126
00:09:11.120 --> 00:09:12.720
Here's what the web page looks like.

s127
00:09:12.720 --> 00:09:14.360
This is a modern website.

s128
00:09:14.360 --> 00:09:16.440
You know, it's nba.com.

s129
00:09:16.440 --> 00:09:19.920
You and I can see the score, 125 versus 109.

s130
00:09:19.920 --> 00:09:21.000
Okay.

s131
00:09:21.000 --> 00:09:28.120
Uh behind the scenes, when you actually load the page and read it, this is what initially gets loaded.

s132
00:09:28.120 --> 00:09:30.800
There's an empty placeholder initially.

s133
00:09:30.800 --> 00:09:41.480
And you wait a few hundred milliseconds to a few seconds, depending on your network connection, and your browser makes an asynchronous call later to fetch the information.

s134
00:09:41.480 --> 00:09:47.800
So, the browser makes a call to an endpoint that it's extracting that information from.

s135
00:09:47.800 --> 00:09:51.520
That endpoint responds with the JSON that contains the answer.

s136
00:09:51.520 --> 00:10:00.000
If you just read the HTML when you load it, the answer is not in the HTML, and so your your chatbot doesn't have access to that, either.

s137
00:10:00.000 --> 00:10:02.400
So, okay, you go, "Fine, these are just details.

s138
00:10:02.400 --> 00:10:05.400
I just have to add some weights and sleeps, and sure."

s139
00:10:05.400 --> 00:10:06.440
Okay, fair enough.

s140
00:10:06.440 --> 00:10:09.200
I will take you to another example.

s141
00:10:09.200 --> 00:10:15.920
Um I ask you the question on a product webpage, "Is the 25-mm osmium cube in stock or out of stock?"

s142
00:10:15.920 --> 00:10:18.080
You're on a on this product website.

s143
00:10:18.080 --> 00:10:21.160
Um you scroll down, there is a drop down.

s144
00:10:21.160 --> 00:10:28.920
Uh that drop down is telling you and I as humans, you know, three things are sold out, one thing is in stock, even though it doesn't actually say in stock.

s145
00:10:28.920 --> 00:10:40.400
Like, there's no text there that says in stock, but you understand that grayed out means sold out, and you know, sometimes the sold out won't actually be there as a text, sometimes it'll just be unclickable grayed out.

s146
00:10:40.400 --> 00:10:43.240
Okay, surely this information must be in the code somewhere.

s147
00:10:43.240 --> 00:10:47.200
You go and read the HTML, and it turns out there is an option selector.

s148
00:10:47.200 --> 00:10:50.920
It actually doesn't say any of the things that I'm seeing on screen.

s149
00:10:50.920 --> 00:10:53.160
It doesn't say sold out.

s150
00:10:53.160 --> 00:10:54.800
It doesn't say available.

s151
00:10:54.800 --> 00:10:56.200
So, what's going on?

s152
00:10:56.200 --> 00:11:05.000
Uh turns out behind the scenes, uh the browser makes a call, gets a JSON object, which is the variable, which is product count.

s153
00:11:05.000 --> 00:11:07.160
That contains a variable called quantity.

s154
00:11:07.160 --> 00:11:10.839
That quantity is, you know, 10 sometimes, zero sometimes.

s155
00:11:10.839 --> 00:11:15.520
That's just, you know, how many things can the back end support right now.

s156
00:11:15.520 --> 00:11:23.600
Some of those quantities are zero, and there's a different rendering script that, anytime they're zero, grays it out and makes it unclickable.

s157
00:11:23.920 --> 00:11:33.920
Fundamentally, what is happening here is this information that you are seeing on screen is not written somewhere as pure text.

s158
00:11:33.920 --> 00:11:35.520
It is calculated.

s159
00:11:35.520 --> 00:11:36.960
It is rendered.

s160
00:11:36.960 --> 00:11:40.240
Um and for people who work in the browser industry, they understand this.

s161
00:11:40.240 --> 00:11:46.920
But, you know, often people who are coming from an AI background like me, uh we didn't always understand this.

s162
00:11:46.920 --> 00:11:49.400
The browser is a rendering engine.

s163
00:11:49.400 --> 00:11:51.960
You are seeing pixels on screen.

s164
00:11:51.960 --> 00:11:54.320
It's Think of it as a game engine.

s165
00:11:54.320 --> 00:12:00.440
Um and you're asking, "Can I not read the source code of the game and predict exactly what the pixels are going to be?"

s166
00:12:00.440 --> 00:12:08.160
Well, yes, eventually, but right now you're asking for an exact inversion of that process.

s167
00:12:08.280 --> 00:12:12.600
Fundamentally, the web was built for human eyes.

s168
00:12:12.600 --> 00:12:17.760
Pixels are the source of the truth because the consumers of the websites are humans.

s169
00:12:17.760 --> 00:12:20.080
That is what it was built for.

s170
00:12:20.080 --> 00:12:31.240
And so, there is a implication here that we have to we have to grapple with, which is machines will need vision to operate those things because the web was was built for for human consumption.

s171
00:12:31.240 --> 00:12:44.640
Um in a way, this is the bitter lesson uh for web agents that the more you end up writing scaffolds around existing websites, the it doesn't actually generalize to the long tail of the web.

s172
00:12:44.640 --> 00:12:52.253
The thing that generalizes is the thing that it was designed for, which is the most general solution, just pixels in.

s173
00:12:52.253 --> 00:12:53.240
[snorts]

s174
00:12:53.240 --> 00:12:58.800
This is what um you know, we and some others in the in the area have been working on.

s175
00:12:58.800 --> 00:13:01.880
Um we have a model called Navigator.

s176
00:13:01.880 --> 00:13:05.120
Um the first version of the model went out in November last year.

s177
00:13:05.120 --> 00:13:07.760
The first version acted purely like a human.

s178
00:13:07.760 --> 00:13:11.560
Screenshot in, button clicks and scrolls out.

s179
00:13:11.560 --> 00:13:16.600
Um I will tell you in a second that is not where you should settle on, but it is a general solution.

s180
00:13:16.600 --> 00:13:18.680
It lets you do things like this.

s181
00:13:18.680 --> 00:13:23.920
Um I give you an e-commerce website, and I tell you there is this discount code.

s182
00:13:23.920 --> 00:13:27.200
Please go The discount code is applicable under certain constraints.

s183
00:13:27.200 --> 00:13:33.240
Maybe it's only on a product, maybe it's only on a certain set of dates, maybe it's only um with this minimum cart threshold.

s184
00:13:33.240 --> 00:13:38.240
I describe that in natural language and I tell you, "Tell me if this discount code is valid or not."

s185
00:13:38.240 --> 00:13:40.000
There is no API for this.

s186
00:13:40.000 --> 00:13:52.560
The reason The way to do it is just like a human can, you go to that website, you find the product that is described, you add it to cart, you apply the discount code, and you check whether the claim of 22%

s187
00:13:52.560 --> 00:13:54.880
off was met or not.

s188
00:13:54.880 --> 00:13:59.480
And that is, you know, screenshot in, button click out.

s189
00:13:59.480 --> 00:14:05.080
This trajectory took, you know, 20, 30, 40 steps, depending on the sophistication of the task.

s190
00:14:05.080 --> 00:14:10.640
If you can do it on your browser, uh this model can accomplish it in principle.

s191
00:14:10.640 --> 00:14:21.334
In practice, of course, there are accuracy gaps and so on, but in principle, this task is solvable, whereas in a lot of earlier cases, even in principle, that task may not be solvable.

s192
00:14:21.334 --> 00:14:22.920
[snorts]

s193
00:14:22.920 --> 00:14:29.720
So, my claim is the web was built for human eyes, machines will need vision, but of course, they do not need to be limited to human ways.

s194
00:14:29.720 --> 00:14:36.080
Just because for the long tail, you need to have a capability, it does not mean that is the only way you should do it.

s195
00:14:36.080 --> 00:14:38.720
Um here is an example showing that.

s196
00:14:38.720 --> 00:14:43.480
The next version of the model that we trained can also write JavaScript on demand.

s197
00:14:43.480 --> 00:14:45.400
So, this is a Chrome extension.

s198
00:14:45.400 --> 00:14:50.480
On the right, you see an action that says execute JS, value default, text select.

s199
00:14:50.480 --> 00:14:55.440
On the left, you saw that the model filled out multiple form fields simultaneously.

s200
00:14:55.440 --> 00:14:58.920
The reason why it could do that is because it wrote a little bit of a function.

s201
00:14:58.920 --> 00:15:11.560
It can read the code when necessary, it can write code when necessary, because the browser, after all, is an engine that can execute code, but it has a sort of formal verification system built in.

s202
00:15:11.560 --> 00:15:16.280
It It is seeing the screenshot, that is the source of the truth, so it knows whether it succeeded or not.

s203
00:15:16.280 --> 00:15:25.525
So, click buttons when you have to, write code when you have to, and look at the result uh through pixels, because that is the that is the source of of truth.

s204
00:15:25.525 --> 00:15:26.640
[snorts]

s205
00:15:26.640 --> 00:15:31.400
And of course because these things are are machines, you can string them into multi-agent systems.

s206
00:15:31.400 --> 00:15:38.320
So, you can have an orchestrator that is launching multiple navigators in parallel each with a cloud cloud sandbox instance.

s207
00:15:38.320 --> 00:15:40.480
They are clicking buttons on multiple websites.

s208
00:15:40.480 --> 00:15:48.160
So, you can accomplish things that would be superhuman because no human would be able to paralyze over that many instances.

s209
00:15:48.720 --> 00:15:58.520
Um around here usually, you know, in this flow of an argument is when people start asking are computer use models actually good enough for these tasks?

s210
00:15:58.520 --> 00:16:08.600
There's actually a perception online that it's not clear whether progress on computer use has been fast and there are questions about why has progress been slow?

s211
00:16:08.600 --> 00:16:14.760
That's not the reality I'm seeing and that's not the reality that the numbers back up.

s212
00:16:14.760 --> 00:16:18.680
This is a popular benchmark online mind to web.

s213
00:16:18.680 --> 00:16:19.960
No benchmark is perfect.

s214
00:16:19.960 --> 00:16:27.120
The point isn't that this is the the right solution, but on the x-axis are release times of different models.

s215
00:16:27.120 --> 00:16:31.560
On the y-axis is performance which is human eval on on this.

s216
00:16:31.560 --> 00:16:36.040
So, a human went in, looked at the trajectory, decided whether it was correct or not.

s217
00:16:36.040 --> 00:16:39.880
And basically this particular version of the benchmark is saturated.

s218
00:16:39.880 --> 00:16:45.800
The last the model that we just released navigator N1.5 is sitting at 97% human eval.

s219
00:16:45.800 --> 00:16:48.280
Eight trajectories out of 300 are incorrect.

s220
00:16:48.280 --> 00:16:51.960
At this point of time, you should just retire the benchmark, build something harder.

s221
00:16:51.960 --> 00:16:55.640
There's about, you know, 30 to 50 steps of interaction that are happening.

s222
00:16:55.640 --> 00:16:57.640
The next step is to go for something larger.

s223
00:16:57.640 --> 00:17:05.000
So, at least in numbers what I'm seeing, we're seeing steady progress in computer use agents becoming being able to do more and more things.

s224
00:17:05.000 --> 00:17:09.671
And this is the model that I showed pixels in button clicks and code out.

s225
00:17:09.671 --> 00:17:10.760
[snorts]

s226
00:17:10.760 --> 00:17:18.439
Around this time usually is when people start asking questions, okay, so they're getting better, but aren't these things slow?

s227
00:17:18.439 --> 00:17:28.600
Cuz after all, you're looking at a screen, you're you're clicking a button, there are lots of buttons to click, and you know, these things are expensive if you're running them for hundreds of of things.

s228
00:17:28.600 --> 00:17:31.680
Um that claim I think is largely true.

s229
00:17:31.680 --> 00:17:33.280
There is some truth to it.

s230
00:17:33.280 --> 00:17:39.240
Um but I think people forget how how much you can optimize these things out.

s231
00:17:39.240 --> 00:17:48.840
Um so this is our results compared to the frontier models, Opus 4.7, GPT 5.5, on a couple of Brazilians benchmarks.

s232
00:17:48.840 --> 00:17:54.320
Um we're slightly better, but I think that's within statistical uh noise in terms of accuracy.

s233
00:17:54.320 --> 00:17:57.000
That improvement I wouldn't beat the drum on.

s234
00:17:57.000 --> 00:18:01.560
What I would emphasize is latency per step and cost per task.

s235
00:18:01.560 --> 00:18:10.840
In terms of latency, this is a smaller footprint model, that's why it's a lot faster than some of the trillion parameter plus models, and there are corresponding cost savings.

s236
00:18:10.840 --> 00:18:21.320
So if you have something like on these data sets, something like 20, 30 steps of interaction, you're looking at 80 cents per task versus $2.30, and that makes a big difference.

s237
00:18:21.320 --> 00:18:27.720
And so the models are getting cheaper uh in that sense, and you can launch them at at scale.

s238
00:18:27.880 --> 00:18:36.200
Um so hopefully at a at a high level, I've sort of convinced you that there is something off with this argument.

s239
00:18:36.200 --> 00:18:39.920
You know, this is where where what my goal was, this is where I started.

s240
00:18:39.920 --> 00:18:44.240
AI agents are going to be the dry primary drivers of action.

s241
00:18:44.240 --> 00:18:53.760
Um that seems sort of an uncontestable statement because, you know, the underlying intelligence of the models is becoming larger and larger.

s242
00:18:53.760 --> 00:18:59.720
Um there is a certain gain of productivity and efficiency and, you know, just ease of life that you get, so it makes sense.

s243
00:18:59.720 --> 00:19:15.440
But I think this hypothesis that suddenly overnight 30 years of infrastructure that was built layer upon layer for human consumption will, in what, two, 5, 10 years be reinvented

s244
00:19:15.440 --> 00:19:18.120
is I think a a fantasy.

s245
00:19:18.120 --> 00:19:21.640
Um But this is ultimately what you want, right?

s246
00:19:21.640 --> 00:19:32.280
Ultimately, you want an endpoint that um you know, some higher-level entity can go to and I say, I want you to do X. There is some task.

s247
00:19:32.280 --> 00:19:35.800
Maybe I give you uh that task description in natural language.

s248
00:19:35.800 --> 00:19:39.480
Maybe I have some programmatic description with parameters.

s249
00:19:39.480 --> 00:19:41.640
And there's you know, some endpoint I should be able to do it.

s250
00:19:41.640 --> 00:19:44.120
So, how will the web be agentified?

s251
00:19:44.120 --> 00:19:46.800
My answer is just extrapolate this trend.

s252
00:19:46.800 --> 00:19:48.360
This is where we stand today.

s253
00:19:48.360 --> 00:19:50.240
Accuracies are getting higher.

s254
00:19:50.240 --> 00:19:52.040
Benchmarks are falling.

s255
00:19:52.040 --> 00:19:53.720
Latencies are getting smaller.

s256
00:19:53.720 --> 00:19:55.200
Costs are falling.

s257
00:19:55.200 --> 00:19:57.800
And that trend, I think, will continue.

s258
00:19:57.800 --> 00:20:04.320
And the way we will agentify the web and in in in some sense, this is depressing, but this is true.

s259
00:20:04.320 --> 00:20:10.280
We will just pile on another layer of mess on top of the mess that the web is.

s260
00:20:10.280 --> 00:20:14.400
Which Which is to say, you know, you will issue a task.

s261
00:20:14.400 --> 00:20:21.600
Behind the scenes, there will be hundreds of browsers that are pretending and clicking buttons just like humans would.

s262
00:20:21.600 --> 00:20:25.000
Um And they will give you a result in a structured format.

s263
00:20:25.000 --> 00:20:27.560
It will cost less than a penny.

s264
00:20:27.560 --> 00:20:29.920
It will run in your browser sometimes.

s265
00:20:29.920 --> 00:20:33.000
It will, you know, run in less than 100 milliseconds.

s266
00:20:33.000 --> 00:20:35.760
And at some point, you will say, "Yeah, that's an API.

s267
00:20:35.760 --> 00:20:37.440
Like, why do I care?"

s268
00:20:37.440 --> 00:20:39.280
And that's how we'll we'll agentify the web.

s269
00:20:39.280 --> 00:20:41.800
So, thank you.

s270
00:20:41.894 --> 00:20:43.894
[applause]