WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=ZyIoTOAbRfs

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/ZyIoTOAbRfs.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:12.560 --> 00:00:20.080
Good to give this talk and just want to say right off the bat this is probably going to be a little different from what you've seen so far at AI conference.

s2
00:00:20.080 --> 00:00:28.160
I'm here to not deliver an agenda on any company's behalf but just to expose a lot of alpha in data markets.

s3
00:00:28.160 --> 00:00:38.800
But also just tell you what's really going on behind the scenes in a in a very murky landscape where nobody seems to know how Remarque handshake and a lot of these folks actually produce data.

s4
00:00:38.800 --> 00:00:45.000
So you know, quick reframe before we start when people hear data markets they picture scale around 2019.

s5
00:00:45.000 --> 00:00:55.000
They picture these rooms of annotators in like Manila labeling images and that's that's real and it's maybe you know, 10 to 15 billion dollars a year per lab but it's kind of the least interesting part.

s6
00:00:55.000 --> 00:01:02.240
The models work now what scares and badly priced is the sort of data that takes some from generalist competence into real expertise.

s7
00:01:02.240 --> 00:01:09.600
We knew this since 2024 when Scale got acquired and we were spamming a lot of GPQA data sets.

s8
00:01:09.600 --> 00:01:13.040
So look, let me start with this framing.

s9
00:01:13.040 --> 00:01:20.160
Data is to the white collar revolution what coal and iron basically is to the Victorian age and the information age right now.

s10
00:01:20.160 --> 00:01:23.920
I have this piece called the TAM is in a vertical it's all of labor.

s11
00:01:23.920 --> 00:01:37.120
And so the supply chain is basically just doing what every sort of industrializing supply chain does it unbundles two years ago when vertically integrated giant like a Remarque or Surge or Scale AI if you

s12
00:01:37.120 --> 00:01:39.560
if you guys are unfamiliar they're massive data companies.

s13
00:01:39.560 --> 00:01:46.160
They had to do all of it but that was because that's the only way that unit economics worked in an immature market today.

s14
00:01:46.160 --> 00:01:48.320
That's increasingly not the case.

s15
00:01:48.320 --> 00:01:57.160
Specialists outcompete the giants at a lot of steps sourcing the people, building environments, designing rewards, running evals.

s16
00:01:57.160 --> 00:01:59.680
The fragmentation I would say, is pretty permanent.

s17
00:01:59.680 --> 00:02:01.000
It's not transitional.

s18
00:02:01.000 --> 00:02:11.959
And uh quality increasingly does not scale linearly with quantity, which leads to a sort of cottage industry in data land right now, where you have labs literally mandates uh vendor diversification

s19
00:02:11.959 --> 00:02:19.560
on the scale of like 20 to 30 different vendors because they inherently distrust their ability to scale quality with quantity.

s20
00:02:19.840 --> 00:02:26.280
So, to orient you, you know, here's here's the sort of argument as it developed through this year.

s21
00:02:26.280 --> 00:02:30.120
Um January, we saw a lot of industrialization and unbundling.

s22
00:02:30.120 --> 00:02:38.000
You'll hear me come back to this mechanism I call the Antikythera mechanisms, which is this sort of bespoke systems that translate messy business context into Evals.

s23
00:02:38.000 --> 00:02:43.959
Uh increasingly important in labs' hungry quest for real-world seed data to set at seed ends.

s24
00:02:43.959 --> 00:02:54.480
Um I also explored this concept called type one versus type two data, uh contrived versus non-contrived for the more researcher types in the audience, and why the GPQA cell playbook that works in 2024

s25
00:02:54.480 --> 00:02:58.560
for data acquisition kind of falls apart on long horizon unverifiable work.

s26
00:02:58.560 --> 00:03:08.640
Um so, you know, going through all these topics, all of which are online, uh I'll also I'll actually just skip to the more interesting part, but I'm bringing this up just here

s27
00:03:08.640 --> 00:03:14.720
in case any of the particular topics I talk about are are particularly interesting and you want to dive in more.

s28
00:03:14.720 --> 00:03:17.640
So, model improvement is a function of three inputs.

s29
00:03:17.640 --> 00:03:22.440
We all sort of see this uh commonly expressed as dodge, compute, data, and talent.

s30
00:03:22.440 --> 00:03:36.480
I put together this very uh like rudimentary sort of chart online in one of my pieces and a very rudimentary uh equation just to express the fact that, you know, if there's any sort of imbalance in this compute, data, and talent, then you start seeing

s31
00:03:36.480 --> 00:03:42.040
uh a sort of inefficiency in producing uh what I call like generalized AI model performance, right?

s32
00:03:42.040 --> 00:03:49.519
But also, uh we we see a sort of inefficiency in in CapEx spend that it that is a sort of result of this equation right now.

s33
00:03:49.519 --> 00:03:57.320
Um exponentially increasing CapEx spend, but sort of like um AI revenues are are sort of falling far behind.

s34
00:03:57.320 --> 00:04:00.040
Uh data is sort of the underfunded leg here.

s35
00:04:00.040 --> 00:04:02.760
It's the one that sort of turns a generalist model into a real expert.

s36
00:04:02.760 --> 00:04:15.480
It's the one that's uh actually I believe quite lacking in this equation and thus because of the imbalance penalty parameter concepts that I'm expressing, um presents a whole opportunity.

s37
00:04:15.560 --> 00:04:20.600
But you know, going back to what I said at the start, what is data actually?

s38
00:04:20.600 --> 00:04:26.680
So, most people picture state-based data, the rows in an ERP, which is like the final output or saved file.

s39
00:04:26.680 --> 00:04:33.800
That's kind of the 2023 like next token prediction model um and it's mostly personal data wrapped in privacy law.

s40
00:04:33.800 --> 00:04:41.440
What is actually available nowadays is process-based data, which is the trajectory, the reasoning trade trace, the sequence of decisions.

s41
00:04:41.440 --> 00:04:48.919
Um so, it it's kind of what gets a professional to get from a blank page to a finished work output and sort of delineates how the work gets done.

s42
00:04:48.919 --> 00:04:51.760
Um so, on top of that sits a quality axis.

s43
00:04:51.760 --> 00:04:53.000
This is the vocabulary.

s44
00:04:53.000 --> 00:04:54.080
I'll use all talk.

s45
00:04:54.080 --> 00:05:05.560
Um type type one data is a sort of pure capture of real workflows like GitHub commits or session replays with minimal reward shaping by non-experts and type two is contrived data, where you sort of hire experts,

s46
00:05:05.560 --> 00:05:09.560
you sit them in an arbitrary setting, you have them manufacture examples.

s47
00:05:09.560 --> 00:05:19.200
Type two, right place to start models when they were reading at a first grade level, I think anybody in the world could teach them as a third grade teacher.

s48
00:05:19.200 --> 00:05:30.480
But type one is what gets you from 20 to 80% so to say cuz the real the realism is inherited from the work itself and the structural reason why it matters is the data is kind of the most appreciable asset there is.

s49
00:05:30.480 --> 00:05:35.080
Um a data set is only available in so far as frontier data markets as the frontier moves.

s50
00:05:35.080 --> 00:05:43.640
So, the only durable supply of it technically is a live business you partner with, not a dead startup's code bases like so many data companies out there are buying today.

s51
00:05:43.640 --> 00:05:51.720
The dirty secret of the industry though is that everybody sells um type two and bills it as type one.

s52
00:05:52.120 --> 00:06:01.200
So, now let me talk about the central axis of how you can think about the sort of verification and deciding which application layer domains are are like maturing first and why.

s53
00:06:01.200 --> 00:06:11.800
Uh you know, there's a reason why cloud science came out so far um you know, now in the future after models got good before a lot of other application layer advances.

s54
00:06:11.800 --> 00:06:15.040
So, Jason Wei is a researcher online whose blog is great.

s55
00:06:15.040 --> 00:06:15.840
You should read him.

s56
00:06:15.840 --> 00:06:16.680
He has this law.

s57
00:06:16.680 --> 00:06:18.200
It's called Verifier's Law.

s58
00:06:18.200 --> 00:06:22.080
The ease of training a model to do a task is proportional to how verifiable the task is.

s59
00:06:22.080 --> 00:06:25.640
So, I break verifiability to three axes here.

s60
00:06:25.640 --> 00:06:28.400
And once you have them, a huge amount of this market stops being mysterious.

s61
00:06:28.400 --> 00:06:30.560
First, asymmetry of verification.

s62
00:06:30.560 --> 00:06:33.600
How hard is it to decompose a task into checkable steps?

s63
00:06:33.600 --> 00:06:34.680
Veracity of verification.

s64
00:06:34.680 --> 00:06:37.880
How much consensus is there about what correct even means?

s65
00:06:37.880 --> 00:06:39.800
And then thirdly, proliferation of verification.

s66
00:06:39.800 --> 00:06:44.200
How often does the real world sort of hand you fresh examples of verified work?

s67
00:06:44.200 --> 00:06:55.080
If you think about like coding as the first mature AI app layer market, that's really no accident because we were blessed to have something called GitHub from web 2.0 which solved all three of these at once.

s68
00:06:55.080 --> 00:06:58.120
Unit tests, they sort of give you objective decomposable correctness.

s69
00:06:58.120 --> 00:07:02.280
The community agrees on what working code means generally.

s70
00:07:02.280 --> 00:07:08.520
And there are effectively infinite public examples with these commit messages as basically free reasoning traces.

s71
00:07:08.520 --> 00:07:11.520
So, they score high, high, and high on these three axes.

s72
00:07:11.520 --> 00:07:15.000
Now, look where the money is trying to go now.

s73
00:07:15.000 --> 00:07:18.040
Biology, security, taste, finance, health care, law.

s74
00:07:18.040 --> 00:07:25.160
These sit pretty low on veracity, pretty low on verification, and the verification examples are sort of like locked all in these enterprise workflows.

s75
00:07:25.160 --> 00:07:29.160
No web 2.0 system ever captured really or very sparingly few.

s76
00:07:29.160 --> 00:07:37.160
Uh that's why, you know, probably maybe you've been approached by Mercari to buy your Slack logs or your Jira logs as of late if you work at an AI company.

s77
00:07:37.160 --> 00:07:38.440
And that's the whole game, right?

s78
00:07:38.440 --> 00:07:39.440
It's it's predictive.

s79
00:07:39.440 --> 00:07:40.640
It's not just descriptive.

s80
00:07:40.640 --> 00:07:46.000
Classify any professions like tasks on these three axes, and you can sort of tell which markets mature most.

s81
00:07:46.000 --> 00:07:52.400
So, it's no surprise that after code, we went to search, and after search, we went to finance, and after finance, we went to healthcare and law.

s82
00:07:52.400 --> 00:07:59.040
And after healthcare and a law, well, I'd say cyber, biological, uh and scientific discovery.

s83
00:07:59.040 --> 00:08:03.960
And maybe even taste, which is probably the most unverifiable out of all these.

s84
00:08:03.960 --> 00:08:06.320
So, a verification is a sort of bottleneck.

s85
00:08:06.320 --> 00:08:13.680
Um let me talk about why the industry sells a lot of snake oil here, and why most benchmarks you see are are sort of quietly fake.

s86
00:08:13.680 --> 00:08:17.960
Um the dominant type two recipe is like let's hire domain experts.

s87
00:08:17.960 --> 00:08:21.720
Um let's have them chat use chat to generate plausible tasks.

s88
00:08:21.720 --> 00:08:29.040
Let's have them solve those tasks, and let's cherry-pick the ones where the model diverged, and let's package them as a hard North Star benchmark.

s89
00:08:29.040 --> 00:08:33.440
And then, this is the perverse part, let's sell the data to hill climb that same benchmark.

s90
00:08:33.440 --> 00:08:38.560
It's basically um and maybe some of you guys here in SF have heard this a little too much.

s91
00:08:38.560 --> 00:08:40.560
It's Goodhart's law with a profit motive.

s92
00:08:40.560 --> 00:08:47.920
Basically, the moment your measure like becomes a target, and then the target is set by people who aren't true domain experts, it stops sort of measuring anything real.

s93
00:08:47.920 --> 00:08:52.760
And so, the whole market, uh I like to call it sits in a sort of fog of war.

s94
00:08:52.760 --> 00:08:56.920
Labs, vendors, and enterprises, they're all kind of guessing which of the data actually improves a model.

s95
00:08:56.920 --> 00:08:58.120
Nobody can see this clearly.

s96
00:08:58.120 --> 00:09:00.760
There's a structural tell.

s97
00:09:00.760 --> 00:09:05.080
Contrived benchmarks, they only ever test a single isolated in-distribution question.

s98
00:09:05.080 --> 00:09:12.800
They can't test whether a model sort of sustains like correct reasoning across a long dependent episode because these tests were all sort of meant to be solved in isolation.

s99
00:09:12.800 --> 00:09:15.200
Uh and may I spent a lot of time in this.

s100
00:09:15.200 --> 00:09:28.440
Um increasingly, uh in a lot of Anthropic blog posts, but also like a lot of other researchers have noted that cross-harness differencing and cross-infrastructure differencing is the primary cause for a lot of benchmark divergences in performance.

s101
00:09:28.440 --> 00:09:34.800
A lot of you are probably very familiar with like the frontier squeezes of the world and the deep squeezes of the world benchmarks.

s102
00:09:34.800 --> 00:09:43.320
Um there are issues that all of these benchmarks have uh related to that specifically.

s103
00:09:43.320 --> 00:09:52.080
In particular, just high false positive and false negative rates that are not immediately obvious when you look at the above head stats on the benchmark.

s104
00:09:52.080 --> 00:10:00.080
So, a single benchmark number under a single scaffold is like basically one sample from a distribution who's basically with nobody measured.

s105
00:10:00.080 --> 00:10:03.160
And that's why there's so much what I call benchmark psychosis today.

s106
00:10:03.160 --> 00:10:05.600
It's like a it's a pretty noisy sample.

s107
00:10:05.600 --> 00:10:14.560
So, um you know, really briefly on an example I took I I always have like I basically have an internal version of Val's AI and a lot of private benchmarks.

s108
00:10:14.560 --> 00:10:20.160
Um so I I took three finance tasks from three real-world vendors.

s109
00:10:20.160 --> 00:10:26.760
Um like an ARR waterfall reconciliation and LBO valuation memo and a sort of like long short pair trade for hedge fund uh trading.

s110
00:10:26.760 --> 00:10:34.600
So, it's a long horizon non-verifiable finance tasks, relatively robust like deterministic verifiers paired with LLM as a judge uh applied correctly.

s111
00:10:34.600 --> 00:10:46.120
So, um when you actually run these, I think you'll you'll notice very clearly Opus 4.8 is worse than 4.7 on a lot of these rubrics.

s112
00:10:46.120 --> 00:10:53.400
Uh you'll you'll start noticing things if you actually do rubric analysis that the 4.7 to 4.8 sub over-engineered self-reflection.

s113
00:10:53.400 --> 00:11:00.120
Um you'll you'll notice that uh GPT 5.5 and Opus 4.08 score within three points on the same task.

s114
00:11:00.120 --> 00:11:14.080
They fell in like exactly opposite directions, whereas GPT nails the arithmetic um but Opus nails the the methodology but loses arithmetic, which is to say uh it's it's clear if you actually do very good agnostic benchmarking,

s115
00:11:14.080 --> 00:11:18.160
um you you can tell where the post-training directions for a lot of these teams went.

s116
00:11:18.160 --> 00:11:23.839
And subsequently, the it informs, I think, like a lot of the data that that comes for this.

s117
00:11:23.839 --> 00:11:35.920
Um so, look, you know, going back to the start, a single benchmark number is a sample from a distribution nobody measured, and it's basically like taking a Swiss Army knife and using the screwdriver bit to cut cheese and like concluding the knife is broken.

s118
00:11:35.920 --> 00:11:45.760
Um so um you know, the the receipts from the last slide, this this goes into an RL environment report that is sent to labs.

s119
00:11:45.760 --> 00:11:58.040
Um but no single leaderboard number ever shows you this part of um you know, data, which some of the most sophisticated RL environment companies, some of which are actually here at this conference,

s120
00:11:58.040 --> 00:12:00.880
um would would show you this.

s121
00:12:00.880 --> 00:12:11.520
So uh how do we read sort of which domain is next instead of chasing hype, and you can actually use this as a proxy data markets as an upstream indicator

s122
00:12:11.520 --> 00:12:15.360
of what next application layer products that labs will come out with.

s123
00:12:15.360 --> 00:12:21.760
In in January, uh Anthropic was spending a lot on cybersecurity data from new vendors.

s124
00:12:21.760 --> 00:12:30.680
And in March and April, they were spending a lot on biological data um from from associated vendors, and what do you know happened like 2 to 3 months after?

s125
00:12:30.680 --> 00:12:35.800
Well, Metis and Cybor and Claude bio / life sciences today.

s126
00:12:35.800 --> 00:12:46.600
Uh So if you want to use this checklist I use, classify the professional's tasks into three axes, apply to long horizon bar, the the ones real labs use, um

s127
00:12:46.600 --> 00:12:55.000
enforced step length heterogeneous tool calls that aren't interchangeable, state transitions that genuinely constrain future actions, mandatory failure recovery.

s128
00:12:55.000 --> 00:13:02.320
Uh you'll notice a pretty mixed bag of of how long horizon is even defined from the vendor perspective in terms of specs.

s129
00:13:02.320 --> 00:13:05.560
Um and then three, you want to look at the raw data for five signals.

s130
00:13:05.560 --> 00:13:12.920
So, sequential decisions against like one entity, an inferable action expert action per step, outcomes recorded by independent parties.

s131
00:13:12.920 --> 00:13:15.720
Um but moreover, just economically available high-wage work.

s132
00:13:15.720 --> 00:13:24.480
So, uh so subsequently, one one counterexample to keep you honest, like robotics, the modality is not settled.

s133
00:13:24.480 --> 00:13:32.680
Um ego versus teleop versus UMI, but moreover, I I find just generally a huge degree of unsophistication within robotics data vendors today.

s134
00:13:32.680 --> 00:13:34.480
A huge degree of unsophistication.

s135
00:13:34.480 --> 00:13:35.440
Yeah.

s136
00:13:35.440 --> 00:13:36.680
Uh I don't know.

s137
00:13:36.680 --> 00:13:38.760
Some of you guys are probably robotics researchers in a crowd.

s138
00:13:38.760 --> 00:13:45.400
I don't know how many times like people have come up to you and they're like, "Oh, here's like 100,000 hours of like iPhone video from my friends in India.

s139
00:13:45.400 --> 00:13:47.929
Do you want to buy this for ego data?"

s140
00:13:47.929 --> 00:13:48.200
[laughter]

s141
00:13:48.200 --> 00:13:55.000
Um so uh in a in in that sort of domain your vendors uh choices entangled with an unsolved research question.

s142
00:13:55.000 --> 00:13:59.960
At the end of the day, you have to realize like all environment companies if they actually succeed, they're more so research accelerators.

s143
00:13:59.960 --> 00:14:01.400
It's a boutique industry.

s144
00:14:01.400 --> 00:14:14.120
If it's venture scalable, it's because the infrastructure they're building agnostically in house helps an enterprise application layer use case rather than assuming that data markets today will stay as they will forever.

s145
00:14:14.120 --> 00:14:17.960
So, don't die in a modality hill, basically.

s146
00:14:18.120 --> 00:14:20.120
So, so one last exhibit just for my work.

s147
00:14:20.120 --> 00:14:22.520
This is the map under everything I just described.

s148
00:14:22.520 --> 00:14:25.080
Look, this share of the white-collar work is sort of on the vertical axis.

s149
00:14:25.080 --> 00:14:26.760
This task horizon is on the horizontal.

s150
00:14:26.760 --> 00:14:33.240
Short horizon is a is very addressable right now, but you think about this long tail on the right side, this deep dependent long horizon work.

s151
00:14:33.240 --> 00:14:36.880
That's where the real economic value and data build out sort of both lives.

s152
00:14:36.880 --> 00:14:43.760
And this threshold line that kind of kind of moves rightward every time somebody builds a real-world data pipeline.

s153
00:14:44.240 --> 00:14:47.280
Um now I want to talk a bit more about the model layer.

s154
00:14:47.280 --> 00:14:50.800
Um it changes who needs to build what.

s155
00:14:50.800 --> 00:14:57.280
So, historical fact, no pioneer of an infrastructure technology has actually held more than 10% of the market in the long run.

s156
00:14:57.280 --> 00:15:04.800
Um I'm not saying that this applies to Anthropic, but you know, I'm just those who forget history are condemned to repeat it.

s157
00:15:04.800 --> 00:15:06.280
Railroads built company towns.

s158
00:15:06.280 --> 00:15:07.440
They charged tyrannical rents.

s159
00:15:07.440 --> 00:15:09.839
They got nationalized as soon as the automobile moved in.

s160
00:15:09.839 --> 00:15:14.600
AWS and Google that consolidated the infrastructure layer and still never captured the application layer.

s161
00:15:14.600 --> 00:15:24.160
And right now, OpenAI and Anthropic are carving out these fiefdoms, but like all these pressures from anti-distillation, export appeals, enterprise exclusivity.

s162
00:15:24.160 --> 00:15:27.320
They're basically the equivalent of real-world brands in their road.

s163
00:15:27.320 --> 00:15:27.600
Right?

s164
00:15:27.600 --> 00:15:36.120
Like the automobile in some cases has already arrived like GLM 5.2 surpassing GPT on a lot of real-world rubrics.

s165
00:15:36.120 --> 00:15:42.120
Um is is is pretty hard proof that a lot of app layer companies can decouple themselves from the model layer.

s166
00:15:42.120 --> 00:15:47.840
And because models differ on efficiency and modality, they're not fungible like electricity.

s167
00:15:47.840 --> 00:15:52.840
So, it's not it's not exactly leading to nationalization, but it's definitely not heading to durable lock-in either.

s168
00:15:52.840 --> 00:15:59.600
And the whole question on what to build next hinges on whether a general enterprise can decouple models from foundation model labs.

s169
00:15:59.600 --> 00:16:06.200
Um But luckily, you know, we for a lot of data companies, this is actually where they're headed.

s170
00:16:06.200 --> 00:16:08.120
I came here to give a talk on data markets.

s171
00:16:08.120 --> 00:16:15.200
I'm here to tell you that that successful data companies nowadays are all pivoting to enterprise.

s172
00:16:16.600 --> 00:16:20.480
I maybe shouldn't say this in a public setting, but I'm recording a handshake.

s173
00:16:20.480 --> 00:16:24.680
People don't notice an incredibly large amount of their revenues are enterprise now.

s174
00:16:24.680 --> 00:16:29.040
Enterprise in ways you wouldn't expect a data business to do.

s175
00:16:29.400 --> 00:16:35.720
Once enterprises stop renting out labs intelligence and starts owning their own, you need an entire abstraction layer that doesn't exist yet.

s176
00:16:35.720 --> 00:16:41.440
There's like five jobs serve and route small models targeted by costly latency and performance profiles.

s177
00:16:41.440 --> 00:16:50.200
Um And uh I to manage your RL data sets instead of like um across base model migration.

s178
00:16:50.200 --> 00:16:55.000
So, when you swap to a new open source base, you rerun post training automatically instead of starting over.

s179
00:16:55.000 --> 00:16:58.680
Three, the Antikythera mechanisms that I talked about before.

s180
00:16:58.680 --> 00:17:04.199
Um and in in the interest of time, happy to talk more about like emerging infrastructure needs afterwards if you want to.

s181
00:17:04.199 --> 00:17:05.640
So, let me bring it all together.

s182
00:17:05.640 --> 00:17:08.959
I think data companies all realize that they have to be neo labs.

s183
00:17:08.959 --> 00:17:15.280
Data businesses do not stay data businesses because the durable value sort of accrues to the services and app layer of actual work.

s184
00:17:15.280 --> 00:17:22.400
So, two takeaways, you know, if you're a researcher, stop outsourcing your definition of realism to the same vendors you buy your eval's and tasks from.

s185
00:17:22.400 --> 00:17:25.079
Um, that's kind of just letting the task test writer grade the task.

s186
00:17:25.079 --> 00:17:29.320
And if you're a builder, your moat's not the data, it's the sort of pipeline into real-world work.

s187
00:17:29.320 --> 00:17:34.560
Plus the infra to keep retraining on it as the models improve underneath you.

s188
00:17:34.560 --> 00:17:36.720
I'll close on this.

s189
00:17:36.720 --> 00:17:39.880
Uh, I'm building um, I'm working on something new.

s190
00:17:39.880 --> 00:17:41.480
This is the first public announcement of it.

s191
00:17:41.480 --> 00:17:43.080
I'm building a ticket through mechanisms.

s192
00:17:43.080 --> 00:17:54.920
I'm working with a lot of real-world companies to monetize their data assets, but moreover, help implement RL as a service with a lot of the enterprises in the world, mitigating a lot of the pitfalls of a lot of the companies I mentioned.

s193
00:17:54.920 --> 00:17:56.960
If you guys want to talk about it afterwards, I'm on Twitter.

s194
00:17:56.960 --> 00:18:03.280
I always write a lot on Twitter and Substack and um, I'm around afterwards, too.

s195
00:18:03.280 --> 00:18:04.692
Thank you.

s196
00:18:04.692 --> 00:18:06.692
[applause]

s197
00:18:18.447 --> 00:18:20.447
[music]