WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=5SRjPT2Eo7A

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/5SRjPT2Eo7A.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:12.680 --> 00:00:13.880
Hey, how's it going?

s2
00:00:13.880 --> 00:00:14.520
I'm Will Brown.

s3
00:00:14.520 --> 00:00:19.600
I lead applied research at Prim and Elect and hope you are all having a a great AI engineer conference.

s4
00:00:19.600 --> 00:00:21.000
This is always one of my favorites.

s5
00:00:21.000 --> 00:00:30.760
It's always just so cool to see what everyone's working on and building and I feel like it is a place where people get to talk about what the future looks like and like start seeing

s6
00:00:30.760 --> 00:00:36.040
the trends in the industry and seeing the progress and what works and what doesn't.

s7
00:00:36.040 --> 00:00:49.440
And that's kind of the theme I want to think about for this talk today is like what comes next from what we're doing right now and what are some of the things that we've been working on and also just been seeing and research that's unfolding

s8
00:00:49.440 --> 00:00:55.920
that kind of points at what we think the future of AI might look like at the organization level.

s9
00:00:55.920 --> 00:01:08.960
Because we've seen a lot of very cool progress in agents and interfaces and bots and things like that over the past several months, years.

s10
00:01:08.960 --> 00:01:11.000
But I there's always like there's always more coming.

s11
00:01:11.000 --> 00:01:20.760
Things are just moving so fast and I think it's a very exciting time to just imagine and speculate and plan for what comes next.

s12
00:01:20.760 --> 00:01:31.640
And so I don't know if any of you remember back in the early ChatGPT days, OpenAI had kind of spelled out their framing of like what are the different stages of

s13
00:01:31.640 --> 00:01:35.640
AI progress on the road to AGI.

s14
00:01:35.640 --> 00:01:41.240
And I think we moved along this faster than we thought we would.

s15
00:01:41.240 --> 00:01:54.160
We like I remember 18 months ago I was here in New York at AI Engineer giving a talk where we had gotten reasoners and chatbots had been around for a while and reasoners were like the new thing.

s16
00:01:54.160 --> 00:01:56.680
But we didn't really have agents yet.

s17
00:01:56.680 --> 00:02:06.492
And now we've climbed pretty quickly through agents and innovators and the question next seems to be, okay, what happens for organizations?

s18
00:02:06.492 --> 00:02:20.212
Like, what does it mean for an organization to truly like be AI native, not just as like a a startup that uses a lot of agents and builds everything with cloud code, but like at the like a large enterprise scale,

s19
00:02:20.212 --> 00:02:32.852
what do companies of the future look like that involve the deep integration of agents throughout the life cycle of the organization, and what do we need to build on the path there?

s20
00:02:32.852 --> 00:02:36.532
Uh so, a bit about Primer and Elect just for some backdrop.

s21
00:02:36.532 --> 00:02:47.972
We build a lot of different things around AI infrastructure for training, deploying, and continuously improving models, and especially open-source models and open-source tooling.

s22
00:02:47.972 --> 00:02:56.412
And so, we do a lot of things, and like we roughly do the sorts of things that you would do inside of a big lab, and these are our products.

s23
00:02:56.412 --> 00:03:10.492
And the reason we do this is because we are imagining a future that is coming pretty quickly, it seems, where open-source models are readily available to train and deploy on your own workflows, on your own data, in your own harnesses,

s24
00:03:10.492 --> 00:03:22.132
and can be the sorts of things that organizations can have grow with them over time that allow compounding their institutional knowledge, their institutional advantage from their workflows, from their data, from their people and their processes,

s25
00:03:22.132 --> 00:03:25.372
in addition to using the models from the frontier labs.

s26
00:03:25.372 --> 00:03:36.532
And so, this talk will be a bit about how we see that evolving and unfolding, where like what is the role of training in an AI native enterprise, how does this coexist with the frontier models,

s27
00:03:36.532 --> 00:03:45.292
and like what do we think is going to be the version of this that really unlocks the most value for people, and what does it look like?

s28
00:03:45.292 --> 00:03:56.012
Um and so, organizations have lots of knowledge, and like the longer organizations been around and the kind of more established it is, the more implicit knowledge there is that kind of doesn't ever get written down.

s29
00:03:56.012 --> 00:03:57.812
Um there are things that do get written down.

s30
00:03:57.812 --> 00:04:00.972
There's codebases, there's emails, there's processes.

s31
00:04:00.972 --> 00:04:02.252
Some of these are very explicit.

s32
00:04:02.252 --> 00:04:05.934
They're like rules of like you must do this, this thing works like that.

s33
00:04:05.934 --> 00:04:14.654
But a lot of things kind of live in people's heads and they live in between the lines of emails, of the meeting notes, of the kind of who talks to who.

s34
00:04:14.654 --> 00:04:24.374
Like a lot of companies will have like our a on paper org chart and then a real org chart of like who actually makes decisions and like how does change get adopted throughout the company,

s35
00:04:24.374 --> 00:04:32.854
what sorts of mandates are likely to take effect, uh and people have kind of built up this knowledge from being a company for a long time and they're like, "Oh, this is kind of how things work.

s36
00:04:32.854 --> 00:04:34.414
This is where the friction points are."

s37
00:04:34.414 --> 00:04:48.574
And I think in many cases, uh and I think this is really one of one of the big opportunities for AI at the organizational level is, is that many individual people are kind of not in a position where they can change the things that are broken in an organization

s38
00:04:48.574 --> 00:04:51.774
um because no one is actually spelling it out.

s39
00:04:51.774 --> 00:04:59.454
No one is is like taking a bird's-eye view of looking at everything that happens in an organization and saying, "Hey, this over here is inefficient, this could be better,

s40
00:04:59.454 --> 00:05:06.854
this thing keeps happening across different teams, but nobody's figured out that the solution is actually been discovered by that team."

s41
00:05:06.854 --> 00:05:09.774
And these are the sorts of things that like agents are great for.

s42
00:05:09.774 --> 00:05:24.814
Like agents are very capable at combing through large amounts of textual data that's unstructured and finding these kinds of patterns and turning them into something useful uh and and surfacing them as like, "Okay, this workflow has a problem because

s43
00:05:24.814 --> 00:05:28.614
it is wasting time and it's inefficient and it could be different instead."

s44
00:05:28.614 --> 00:05:30.774
Um and we don't want to do this once though.

s45
00:05:30.774 --> 00:05:41.254
We want this to kind of become the way that organizations adopt AI broadly where agents are able to get the context they need, but also they're able to improve

s46
00:05:41.254 --> 00:05:43.694
both the agents themselves as well as the organizations.

s47
00:05:43.694 --> 00:05:50.814
And we want agents to be more like our coworkers, but um human coworkers learn on the job.

s48
00:05:50.814 --> 00:06:00.534
Agents currently learn on the job in relatively primitive ways um and they do this via in-context learning um and oops, sorry.

s49
00:06:04.454 --> 00:06:09.415
I'm going to go back a few seconds so we can edit this on the recording.

s50
00:06:09.415 --> 00:06:21.815
Agents learn in relatively primitive ways through like in context learning and this typically looks like them kind of having a skill file written by somebody or like looking around a code base but they don't get the institutional

s51
00:06:21.815 --> 00:06:26.135
level latent knowledge that human employees really do.

s52
00:06:26.135 --> 00:06:27.895
And so they relearn a lot.

s53
00:06:27.895 --> 00:06:29.055
They have to rediscover things.

s54
00:06:29.055 --> 00:06:44.375
They spend a lot of input tokens and I think nowadays people are realizing that like cached input is like the bottleneck in many agent costs and even if you do get great cash rates and cash costs go down, you still have to like reprocess a lot of input to redo a workflow every single time.

s55
00:06:44.375 --> 00:06:52.095
And so to get around this you kind of want to be able to bake in more into the model especially for things that are harder to write down as rules.

s56
00:06:52.095 --> 00:06:59.295
So I I like thinking about the spectrum from like rules to patterns as well as like conceptually like code versus data.

s57
00:06:59.295 --> 00:07:08.335
And so lots of things live as like code and like skills are kind of like this, rules are kind of like this, but things like preferences and friction, heuristics, and trends,

s58
00:07:08.335 --> 00:07:25.255
these are all data and institutions have so much data that they are like running on and the employees of the organization implicitly learn that models never really get a chance to learn and this is the sort of thing that we see as blocking them from becoming true co-workers.

s59
00:07:25.255 --> 00:07:29.375
And so there is a question of like okay, what goes in context, what goes in the weights?

s60
00:07:29.375 --> 00:07:31.095
And part of it is you kind of need to earn it.

s61
00:07:31.095 --> 00:07:33.735
Like things don't just go in the weights right away.

s62
00:07:33.735 --> 00:07:42.015
But if you if you see kind of every new generation of models the models get better not because the prompt is better, but because they train them more.

s63
00:07:42.015 --> 00:07:43.295
Like training is useful.

s64
00:07:43.295 --> 00:07:44.015
Training works.

s65
00:07:44.015 --> 00:07:57.415
Training is why we see the progress we do and the models roughly like trained on lots of data that is aiming to approximate things that makes that the labs expect will be like high value repeatable workflows

s66
00:07:57.415 --> 00:07:59.015
at organizations.

s67
00:07:59.015 --> 00:08:02.815
And I like this like I think there's a process that currently at works.

s68
00:08:02.815 --> 00:08:06.655
It's how the models get better, but if you like spell it out it's relatively inefficient.

s69
00:08:06.655 --> 00:08:18.057
So like the way that a model would learn how to get really good at like using some combination of like Salesforce tools for example, let's say is someone who knows how those tools get used has to be

s70
00:08:18.057 --> 00:08:28.057
looking for work uh and maybe they previously worked at a place that were using these tools and now they are uh browsing kind of expert network sites to make some money in their free time

s71
00:08:28.057 --> 00:08:32.537
and they are sharing their knowledge to someone who works at an expert network company.

s72
00:08:32.537 --> 00:08:41.417
This person at this network company will then this uh take this data from the human and uh pass it to someone else whose job is to create RL environments.

s73
00:08:41.417 --> 00:08:52.817
This RL environment might get sold then to buy someone who doesn't have this expert knowledge to a researcher at a lab who also really isn't the expert in the domain who then will be refining it to put it into

s74
00:08:52.817 --> 00:09:04.337
a training pipeline uh to go into the next frontier model run to make a model that has kind of through this like game of telephone gotten some knowledge of that human originally.

s75
00:09:04.337 --> 00:09:07.657
But, there's so many steps in this pipeline and it's a relatively secret pipeline.

s76
00:09:07.657 --> 00:09:17.497
Like people people talk at across two stages of it, but like the human who knows this thing is not supervising the training run, is not really directly inputting like how the model gets trained

s77
00:09:17.497 --> 00:09:22.257
in the way that a human could if they were doing it for their coding engine that they're working with day-to-day.

s78
00:09:22.257 --> 00:09:37.137
Um and so, what we see as an opportunity and kind of also where we would hope to go is like to close this gap and to allow the person who the same way that when we work with our coding agents we're prompting them like the same things every time telling them to like not repeat the same mistakes.

s79
00:09:37.137 --> 00:09:40.017
Um that this is how you could train models.

s80
00:09:40.017 --> 00:09:50.457
You can tell the models things uh and they will learn these things durably, not just as like a line in your agent's MD which kind of accumulates and then you have to refactor it all the time.

s81
00:09:50.457 --> 00:09:54.897
You can do some of that, but a lot of times what you really want is you're giving kind of contextual preference.

s82
00:09:54.897 --> 00:10:00.777
You're saying in this setting this thing that you did was not exactly what I wanted and I would have preferred this instead.

s83
00:10:00.777 --> 00:10:04.897
And we'd want systems that can compound this into like durable learning.

s84
00:10:04.897 --> 00:10:15.458
Uh and training is a very effective way of doing this because the shape of data you're giving there is quite close to what you would want uh from like an actual training pipeline.

s85
00:10:18.018 --> 00:10:19.658
And so there's there's several different types of learning.

s86
00:10:19.658 --> 00:10:24.218
There's things that people do, there's things that you can do at the harness layer with skills and memory.

s87
00:10:24.218 --> 00:10:32.698
There's knowledge that just lives throughout an organization in file systems and databases that you retrieve with with tools and with code.

s88
00:10:32.698 --> 00:10:45.058
Then there's the models themselves which have this really broad world knowledge, but like we all have I'm sure experienced frustration when using a model whose like cut off of knowledge is like a while back

s89
00:10:45.058 --> 00:10:49.058
and like it you can't really talk to the models about the news of the past 6 months nowadays.

s90
00:10:49.058 --> 00:10:59.218
Like they don't really they have to go like relearn a world model in context every single time and this just does not allow them to really operate as fluently as they can discuss

s91
00:10:59.218 --> 00:11:01.058
things that happened longer ago.

s92
00:11:01.058 --> 00:11:10.778
And so organizations to get the most value out of models, we imagine they will want to increasingly have models that really understand what is going on in the organization

s93
00:11:10.778 --> 00:11:12.138
at an intuitive level.

s94
00:11:12.138 --> 00:11:14.858
And this is the sort of thing that you want to train for.

s95
00:11:14.858 --> 00:11:16.698
Um and so how do you get there?

s96
00:11:16.698 --> 00:11:26.578
Like what is the road for an organization to go from like having this idea in the abstract of like we're going to train models on our data to actually getting there.

s97
00:11:26.578 --> 00:11:30.618
And I think one really useful starting point is evals as essentially test cases.

s98
00:11:30.618 --> 00:11:43.818
I think like with code we've gotten to a point of maturity around like testing and life cycles and software development life cycles for ensuring that code bases grow healthily over time usually.

s99
00:11:43.818 --> 00:11:46.858
And agents are doing so much of this work for us.

s100
00:11:46.858 --> 00:11:58.018
And the version of continual learning that I see I think is like most practical for most organizations is just treating a model kind of like a code base where it is the sort of thing that you can have a life cycle around.

s101
00:11:58.018 --> 00:11:59.578
You can add test cases to.

s102
00:11:59.578 --> 00:12:00.698
You can add features to.

s103
00:12:00.698 --> 00:12:03.218
You can add assets to.

s104
00:12:03.218 --> 00:12:08.698
And this is the sort of thing that the process of using agents like initially you collect traces.

s105
00:12:08.698 --> 00:12:19.979
These traces have your prompts and they have your feedback to things that models did, and this just from the process of using these models, you are accumulating the assets you need to create evals.

s106
00:12:19.979 --> 00:12:36.939
It's just that making evals historically has been a little bit painful, and a lot of organizations have not necessarily wanted to do it manually, but the agents are quite quickly getting to the to the point where thankfully we we have robots that can can do a lot of the eval creation for us, especially as we train them

s107
00:12:36.939 --> 00:12:42.899
more on the best practices of making evals in ways that are robust and amenable to learning.

s108
00:12:42.899 --> 00:12:51.619
Um and so I think another useful uh example here is decision model craze that has kind of been taking the world by storm in the past couple weeks with with Jev,

s109
00:12:51.619 --> 00:13:04.859
and also the parallels around routing and people being very excited about cost optimization because we see that like it's possible to spend very large amounts of money on Claude every month, and we figure out like when should we use a cheaper model?

s110
00:13:04.859 --> 00:13:13.059
Um and this is the sort of thing that I think people are recognizing needs to kind of live in the harness, um but it also is a an attribute that looks relatively similar to

s111
00:13:13.059 --> 00:13:14.059
uh decision models.

s112
00:13:14.059 --> 00:13:20.619
And the reason I bring both these up together is because there's things with very clear labels where there's a a right answer, and there's an opportunity for a human

s113
00:13:20.619 --> 00:13:22.299
or another agent to give feedback.

s114
00:13:22.299 --> 00:13:28.299
You can have the big model and a small model do the same task, judge whether the small model was as good as the big model.

s115
00:13:28.299 --> 00:13:33.739
If so, then we this is now data of like for this kind of thing we can start routing to the small model more.

s116
00:13:33.739 --> 00:13:39.619
Same for a decision model that's learning to kind of mimic the way that a human in a certain situation would make a judgment.

s117
00:13:39.619 --> 00:13:42.099
Uh this is a great labelable action.

s118
00:13:42.099 --> 00:13:52.099
Um where the human might be doing the approval the same way that like we approve our tool calls in Claude code sometimes, but I think most people have turned on like auto uh tool acceptance,

s119
00:13:52.099 --> 00:14:02.259
um but every organization for more bespoke things that aren't just about like tool security are going to have their own rules and preferences, and these are the sorts of things that you could then compound into a decision model.

s120
00:14:02.259 --> 00:14:09.499
And so this is like a very believable form of continual learning that a lot of people could just plug in tomorrow and uh see immediate value from.

s121
00:14:09.499 --> 00:14:19.781
Um but more broadly what we see people wanting to do, and this is a lot of the work that that we do at Prime Electives, working with organizations that are looking to do relatively ambitious training projects

s122
00:14:19.781 --> 00:14:36.861
around turning uh their traces from deploying a model in production for a high-volume use case into rich, robust environments that are blended real and synthetic that kind of capture the complexity of the real task they want to be using long-running agents for.

s123
00:14:36.861 --> 00:14:46.541
And agents are very useful with sufficient orchestration around them, sufficient tooling, and humans in the loop at turning these into environments that are rich enough to train in.

s124
00:14:46.541 --> 00:14:52.261
Uh and we see that this is getting more and more automated with with every new generation of open and closed models.

s125
00:14:52.261 --> 00:15:03.661
And this means that we have an opportunity to start having this whole thing of taking the stuff that we're already doing and turning it into a self-improvement loop just kind of fall out magically from

s126
00:15:03.661 --> 00:15:10.981
applying agents to curation of the data and creating interfaces for humans to to give signal about what they want.

s127
00:15:10.981 --> 00:15:21.621
Um this is a idea that I've been for my own setup been like playing around with a lot that I think will become a a popular interface for interacting with models more in the future.

s128
00:15:21.621 --> 00:15:33.461
You're seeing it a little bit in some of the bots, but I think it especially now that models are good at like writing code on the fly, these kind of widget card features that are like I think a a very practical form of dynamic UI

s129
00:15:33.461 --> 00:15:36.141
that are also really useful for eliciting human decisions.

s130
00:15:36.141 --> 00:15:49.501
So they're kind of like the the tool call approval thing, but much more personalized, much more specific, and also kind of get around the problem of like, "Okay, you have 20 different bots that are going and you have to remember which ones are where

s131
00:15:49.501 --> 00:15:51.261
and who is curating all this for you?"

s132
00:15:51.261 --> 00:15:53.341
And then people are like, "Do you want one bot?"

s133
00:15:53.341 --> 00:15:55.741
And then you're like, "Oh, one's great, but I actually want two."

s134
00:15:55.741 --> 00:15:57.581
Then you have two, then you have 20.

s135
00:15:57.581 --> 00:16:05.141
And I think we kind of are oscillating back and forth between this the single stream and the multi-stream agent threads, but what we really want to do are make decisions.

s136
00:16:05.141 --> 00:16:11.301
We want to give ideas to the agents, let them run with the ideas, and then come back to us when we need to make decisions.

s137
00:16:11.301 --> 00:16:15.461
And the agents hopefully should make good choices about when to bring us decisions.

s138
00:16:15.461 --> 00:16:26.702
And if agents can create interfaces on the fly for us that allow us to very directly give them the information they need, one, this kind of is much more ergonomic for us as humans

s139
00:16:26.702 --> 00:16:32.022
than like reading, trying to parse through a long thread message to see like, "Okay, where was the question?"

s140
00:16:32.022 --> 00:16:39.782
It's also kind of like the human dual side of the the decision model Jeff style, but it also is perfect data.

s141
00:16:39.782 --> 00:16:56.542
And so, it's a setting where we've been experimenting with this as a way of training models to kind of get this feedback loop going from having human input with these kinds of natural interfaces be the sort of thing that directly acts as a label and is the sort of thing that you that a model can directly

s142
00:16:56.542 --> 00:16:59.142
improve on because there is a very explicit target.

s143
00:16:59.142 --> 00:17:04.342
And it it will get around a lot of the problems that we kind of hear about in reward hacking and whatnot.

s144
00:17:04.342 --> 00:17:10.822
Those things are worth addressing, too, but like this is like where we see a very useful starting point being for a lot of these problems.

s145
00:17:10.822 --> 00:17:13.502
And this can be done, doesn't have to be with every model in your stack.

s146
00:17:13.502 --> 00:17:24.542
It can be the orchestration layer models, it can be decision models, but it is an entry point for kind of starting to have this compounding of preferences and latent knowledge happen in the models.

s147
00:17:27.062 --> 00:17:29.222
Um and so, I think there's a recent example today.

s148
00:17:29.222 --> 00:17:35.422
I just added this earlier from a company called a paper instruments that they use some of our tooling to do this.

s149
00:17:35.422 --> 00:17:37.302
They're they're friends of ours.

s150
00:17:37.302 --> 00:17:46.742
But they trained a model for 300K that is kind of Pareto frontier optimal for knowledge work for things like spreadsheets and uh slides.

s151
00:17:46.742 --> 00:17:50.342
And it's like an employee scale investment for an org to do.

s152
00:17:50.342 --> 00:17:55.302
Where like you could like go hire a software engineer or you could like train a model.

s153
00:17:55.302 --> 00:17:59.742
And you can also spread this out over a longer period of time if you're kind of compounding the knowledge over time.

s154
00:17:59.742 --> 00:18:03.342
But it's the sort of thing that is increasingly becoming quite practical.

s155
00:18:03.342 --> 00:18:11.542
And it's the sort of thing that a lot of organizations we see wanting to do now and we kind of see the trend of how much more the demand for this is growing

s156
00:18:11.542 --> 00:18:24.343
month after month uh as people start recognizing one, the value of this and how kind of the the open models are good enough that you can get them to be much better and also much more personalized

s157
00:18:24.343 --> 00:18:30.743
by turning your knowledge for your domain, your harness, your environment into a training setting for them.

s158
00:18:30.743 --> 00:18:35.503
Um, but also it's the sort of thing that naturally lends itself to a a learning loop.

s159
00:18:35.503 --> 00:18:46.943
And so this is a a prototype of something we're doing internally around kind of how we're imagining people can really deploy continual learning in the world, which is that you can have traces, you can start collecting these and you can then

s160
00:18:46.943 --> 00:18:55.863
like every night see the proposed experiments, look at the things that have been flagged as issues and kick off a training run for the night.

s161
00:18:55.863 --> 00:18:58.183
Maybe your inference capacity in the day is serving your agents.

s162
00:18:58.183 --> 00:19:03.543
At night employees are not using them as much and you can use that capacity for training instead.

s163
00:19:03.543 --> 00:19:13.143
And so this is a a very natural way that we see organizations able to like quite soon actually do for real continual learning in the way it sets the models and start

s164
00:19:13.143 --> 00:19:19.223
compounding their knowledge into uh better and better capabilities for their agents.

s165
00:19:19.223 --> 00:19:23.463
Um, and so this is kind of the where we see a lot of this culminating towards.

s166
00:19:23.463 --> 00:19:24.943
There's so many things you do first.

s167
00:19:24.943 --> 00:19:38.383
You discover your workflow, you uh create skills, you make evals, you do routing, and then eventually you start to want to get the economies of scale, get the compounding benefits that have come from the data inside of your companies,

s168
00:19:38.383 --> 00:19:47.303
your organizations, and and have the organizations become increasingly self-improving in ways that uh get better and better over time.

s169
00:19:47.303 --> 00:19:48.903
Um, that's what I got.

s170
00:19:48.903 --> 00:19:52.383
Uh I can maybe take questions if there's any time.

s171
00:19:52.383 --> 00:19:52.783
I don't know.

s172
00:19:52.783 --> 00:19:53.423
No questions.

s173
00:19:53.423 --> 00:19:56.543
Um, and uh yeah, we're at Primer Lex.

s174
00:19:56.543 --> 00:19:58.183
We're working on a lot of this stuff.

s175
00:19:58.183 --> 00:20:06.663
A lot of it is open source and public and we uh blog about it and tweet about it, but also we work with companies and we're hiring and growing and so get in touch if you want to get in touch.

s176
00:20:06.663 --> 00:20:07.833
Thanks.

s177
00:20:07.833 --> 00:20:09.833
[applause]