WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=AQv3qRCG6Gw

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/AQv3qRCG6Gw.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:13.240 --> 00:00:17.400
Uh thanks all for coming to AI engineer and checking out the post training session.

s2
00:00:17.400 --> 00:00:20.760
Um hopefully lots of fun stuff today and throughout the conference.

s3
00:00:20.760 --> 00:00:22.000
Um I'm Will Brown.

s4
00:00:22.000 --> 00:00:28.160
I lead applied research at Primordial AI and today I want to talk about reinforcement learning without verifiable rewards.

s5
00:00:28.160 --> 00:00:41.240
And so many of people may have been learning about RLVR over the past uh year or so uh year and a half as this stuff has really taken off and become the main way that we think about scaling reinforcement learning.

s6
00:00:41.240 --> 00:00:44.400
Um but often we don't actually have verifiable rewards.

s7
00:00:44.400 --> 00:00:48.320
And so messy real world tasks often we're kind of figuring out as we go.

s8
00:00:48.320 --> 00:00:56.120
We're having our agents run around and we kind of in hindsight maybe can like look at what they did and say like okay this is good this is bad.

s9
00:00:56.120 --> 00:01:02.360
Um but sometimes sitting down and just like specifying okay this is the rule this is the goal is not always so straightforward.

s10
00:01:02.360 --> 00:01:11.840
And so this is going to be uh synthesizing a lot of work we've been doing as well as from the broader research literature and some of the things we're building to kind of support

s11
00:01:11.840 --> 00:01:15.280
extending RL into more messy real world tasks.

s12
00:01:15.280 --> 00:01:18.320
And so recap quickly of how reinforcement learning works.

s13
00:01:18.320 --> 00:01:27.040
I would imagine if you're in the post training session here you've probably heard a little bit about RL but for those of you at home and uh for those uh who are kind of still just kind of looking for the crash course

s14
00:01:27.040 --> 00:01:32.480
generally we have an agent which we're going to call a model plus a harness which we place into an environment.

s15
00:01:32.480 --> 00:01:35.960
And so an environment we're going to call a task plus a world.

s16
00:01:35.960 --> 00:01:45.000
A world you could think of as maybe it's a docker image maybe it's a code base maybe it is a collection of uh task specific tools maybe it's some skills

s17
00:01:45.000 --> 00:01:52.880
maybe it is a bunch of applications or browser tabs or things like this as well as a scoring rule uh verifiers or rewards whatever you want to call them

s18
00:01:52.880 --> 00:02:02.160
uh and the agent in the environment are going to interact in a loop and at the end we'll have some reward of how well the agent did in the environment for this task.

s19
00:02:02.160 --> 00:02:13.320
And then reinforcement learning is all about creating an advantage, and the advantage is really about taking the reward minusing some baseline, and maybe doing some scaling, and then [clears throat] now you have a set of

s20
00:02:13.320 --> 00:02:17.920
rollouts from the agent in the environment that you can use to then update the policy.

s21
00:02:17.920 --> 00:02:25.520
The policy here is just the model weights themselves, and the goal here is to take a gradient which nudges the model towards getting higher reward.

s22
00:02:25.520 --> 00:02:41.786
And so all the RL stuff people talk about whether it's GRPO or reinforce or SISPO or any of the other new algorithms people come up with, they're all kind of in this policy gradient framework, which is just about saying, "Okay, how do I make the model do things that have higher reward?"

s23
00:02:41.786 --> 00:02:42.000
[clears throat]

s24
00:02:42.000 --> 00:02:46.240
And so at Primary and Like we build a lot of tooling to power all this at every player.

s25
00:02:46.240 --> 00:02:52.360
We kind of go both from the we start at the compute layer and do lots of large-scale GPU orchestration.

s26
00:02:52.360 --> 00:02:58.440
We build the Primary RL training framework, which powers all of our large-scale reinforcement learning and other algorithms running.

s27
00:02:58.440 --> 00:02:59.760
We build environments.

s28
00:02:59.760 --> 00:03:10.959
We have task sets and harnesses and verifiers as tools that you can mix and match to assemble complex worlds for agents to learn from real-world feedback.

s29
00:03:10.959 --> 00:03:17.760
We have a training platform called Lab, which is anchored around environments where we do both hosted training and evaluations as well as inference.

s30
00:03:17.760 --> 00:03:24.800
And this is to allow people to monitor their experiments and manage the training runs and iterate on their evals and deploy these models.

s31
00:03:24.800 --> 00:03:31.280
And ultimately the models are starting generally from some open-source base model, and you're optimizing it for your task.

s32
00:03:31.280 --> 00:03:46.360
And the goal that we're really trying to enable is more people to be able to become their own research lab, become uh take ownership over the intelligence of their own model weights and that optimize for the task that they care about with themselves as the experts

s33
00:03:46.360 --> 00:03:50.640
steering the model, which means we need to make it way easier for people to do this.

s34
00:03:50.640 --> 00:03:53.320
Currently, for a lot of people, it's still really hard.

s35
00:03:53.320 --> 00:03:57.560
I think you can go like we're all here learning more about how it works because it's hard.

s36
00:03:57.560 --> 00:04:06.920
We don't know how it all works and we're figuring it out as we go in many cases, but we've spent a lot of effort and a lot of time building stuff that hopefully makes this a bit easier for people.

s37
00:04:06.920 --> 00:04:08.200
Um and so what's an environment?

s38
00:04:08.200 --> 00:04:12.000
An environment is tasks, harness, and rewards.

s39
00:04:12.000 --> 00:04:13.600
Uh but it's not just for RL.

s40
00:04:13.600 --> 00:04:18.840
So I think a lot of people think RL when they think environment, but environments and evals are really the same thing.

s41
00:04:18.840 --> 00:04:23.080
You can use these same objects for generating static data, which then you can use for SFT.

s42
00:04:23.080 --> 00:04:26.080
You can do RL or you can do algorithms like on-policy distillation.

s43
00:04:26.080 --> 00:04:32.440
You can do prompt optimization like Jeph I. You can use it as a kind of scientific test bed to iterate on your agents and your harnesses.

s44
00:04:32.440 --> 00:04:39.680
Um and verifiable rewards are the easy case where we just kind of can check exactly was something done correctly or not.

s45
00:04:39.680 --> 00:04:46.080
And so for math, often if you have a numerical answer, you can just parse this out of like a box in the answer from the model and check.

s46
00:04:46.080 --> 00:04:49.560
For code, maybe you want to use test cases or a linter or something like this.

s47
00:04:49.560 --> 00:04:56.640
For tool use, often you have some database state, which you kind of know what you what you're expecting at the end and you can just kind of like check this deterministically.

s48
00:04:56.640 --> 00:05:02.440
And so these are kind of the easy cases where the the uh reward design problem is not so difficult.

s49
00:05:02.440 --> 00:05:05.960
Um but most real-world tasks are not this verifiable.

s50
00:05:05.960 --> 00:05:14.560
Uh for a lot of real agent work tasks, we're having agents do things like write reports that maybe are analyzing a bunch of documents or research.

s51
00:05:14.560 --> 00:05:20.320
Um we're maybe asked to do things like book flights or buy things, but there isn't always like a clean best answer here.

s52
00:05:20.320 --> 00:05:24.800
Uh and there's also notions that are fuzzier like interacting with users like handling a refund.

s53
00:05:24.800 --> 00:05:26.680
Like what does it mean to handle this well?

s54
00:05:26.680 --> 00:05:36.360
Um and so here the signal is less clear and there's a lot of different tricks we might want to explore and techniques we want to develop to ensure that this can be done reliably and scalably.

s55
00:05:36.360 --> 00:05:50.680
Uh and making evals is hard because often times the benchmarks out there that we might like look at in the kind of uh new model releases, it's like a set of a few hundred tasks that a bunch of researchers spend months kind of handcrafting and talking to experts.

s56
00:05:50.680 --> 00:05:57.520
Maybe they worked with data vendors and kind of like spent lots and lots of money kind of getting these to be very like precisely refined.

s57
00:05:57.520 --> 00:06:05.960
And and this isn't very scalable out of the box um especially for things that are more open-ended where there's no kind of clean check for what's good or not.

s58
00:06:05.960 --> 00:06:09.680
Um often the the the real-world situations can be unbound.

s59
00:06:09.680 --> 00:06:13.120
You don't always know what things are going to be in the distribution.

s60
00:06:13.120 --> 00:06:16.040
Like a lot of these cases we are figuring out the distribution as we go.

s61
00:06:16.040 --> 00:06:22.520
And classical machine learning will tell you you can train for the distribution, but generalizing outside of the distribution is kind of an undefined problem.

s62
00:06:22.520 --> 00:06:25.480
And then especially with RL, we have to be very careful about reward hacking.

s63
00:06:25.480 --> 00:06:32.200
And so reward hacking is when you have a kind of loose proxy for your objective that is undefined at the boundaries.

s64
00:06:32.200 --> 00:06:40.880
And then models, if you train with RL, they can learn to exploit this and find weaknesses where there's some path towards a climbing the the reward that doesn't actually

s65
00:06:40.880 --> 00:06:42.360
give you what you want.

s66
00:06:42.360 --> 00:06:54.560
And so really the goal of what we would hope all of this builds into is continual learning, which is a big buzzword that I think a lot of people like talking about in many different ways, but I'm going to use it to mean a very particular thing, which is that

s67
00:06:54.560 --> 00:07:05.919
we want models to be deployed in relatively realistic complex, messy settings and to be able to learn as they go where they are doing things, they are making mistakes,

s68
00:07:05.919 --> 00:07:12.440
they are then able to observe and catch these mistakes after they happen, and use this to not do the same thing again.

s69
00:07:12.440 --> 00:07:16.240
In some cases people want to try to do this at the harness layer of the front layer.

s70
00:07:16.240 --> 00:07:23.760
But ultimately you want a system that can evolve autonomously to be able to get better over time with humans in the loop at that right level of abstraction.

s71
00:07:23.760 --> 00:07:28.960
And I think currently the level of abstraction for doing this is far too low for it to be practical for most people.

s72
00:07:28.960 --> 00:07:34.720
And so this means we need new methods to be able to uh automatize as much of the difficult processes as possible.

s73
00:07:34.720 --> 00:07:36.760
And many of these actually are automatizable.

s74
00:07:36.760 --> 00:07:39.320
They just are kind of difficult problems to solve.

s75
00:07:39.320 --> 00:07:51.000
And so there's a few techniques you can you can use to kind of start making progress here, but one of the goals here is to do online reinforcement learning so that you can kind of iterate on this process as you go, as well as beyond just RL.

s76
00:07:51.000 --> 00:07:56.360
There are other things where you might want to incorporate world knowledge into the model itself, not just in terms of skill refinement.

s77
00:07:56.360 --> 00:08:00.880
RL's great for refining skills, but less so for incorporating like dense new knowledge.

s78
00:08:00.880 --> 00:08:04.480
And so blending these two together is also an important goal.

s79
00:08:04.480 --> 00:08:16.240
And ultimately the what we want from this is to be able to deploy agents into production and have them improve as they go and have the these experiments be monitorable and traceable and replayable so that we can kind of treat

s80
00:08:16.240 --> 00:08:24.960
model optimization very much as a science and make this science accessible to people who have a very wide variety of use cases they want to deploy agents for which don't

s81
00:08:24.960 --> 00:08:27.000
all live in the training distributions of the big models.

s82
00:08:27.000 --> 00:08:32.719
And so we want to be able to do this on top of the best and biggest open models in the world and make this accessible.

s83
00:08:32.719 --> 00:08:34.960
And so how do you manufacture signal?

s84
00:08:34.960 --> 00:08:37.880
There's a bunch of techniques that we've found very useful.

s85
00:08:37.880 --> 00:08:39.080
One is grounding.

s86
00:08:39.080 --> 00:08:46.160
And so grounding roughly means that you have some source material and in machine learning generally you want to have some notion of supervision.

s87
00:08:46.160 --> 00:08:48.200
There's something you're learning from.

s88
00:08:48.200 --> 00:08:56.920
And in messy situations we don't necessarily always have clean supervision, but we can get pretty reliable supervision if we kind of are careful about techniques we use.

s89
00:08:56.920 --> 00:09:08.320
And so grounding is one where you have some source material and the ability to do an AB test of like with and without is a very useful way of creating this kind of capability gap where a model will do better if it has something in context.

s90
00:09:08.320 --> 00:09:12.400
And this gap is something we can exploit to create signal that we can then learn from.

s91
00:09:12.400 --> 00:09:13.720
Judges are also really useful.

s92
00:09:13.720 --> 00:09:18.880
We're relying on the fact that LLMs are already really powerful general reasoners for many things.

s93
00:09:18.880 --> 00:09:27.480
And if we assign the if we set these judges up in the right way, then they can spend computes to make decisions about whether an action was good or bad.

s94
00:09:27.480 --> 00:09:29.120
We also want to be scaling search.

s95
00:09:29.120 --> 00:09:43.240
So search in many ways is something we can apply at many different layers of the pipeline both in terms of creating tasks as well as the worlds as well as the criteria for which we want to be giving judges for answering questions about the quality of a rollout.

s96
00:09:43.240 --> 00:09:49.880
And so for source material, um one very useful version of this, especially for this continual learning goal, is production traces themselves.

s97
00:09:49.880 --> 00:10:01.840
And so what we found is super helpful is taking existing traces from a deployed agent and treating these as the source material where we don't necessarily know up front what the distribution of tasks is, but as an agent is deployed,

s98
00:10:01.840 --> 00:10:08.160
you start collecting more and more examples of let's say user prompts or uh uh calls from an orchestrator agent down into a sub-agent.

s99
00:10:08.160 --> 00:10:10.800
And this starts becoming the distribution.

s100
00:10:10.800 --> 00:10:13.640
We don't have labels yet, but it tells us at least what we want to look for.

s101
00:10:13.640 --> 00:10:19.960
Um and so this is one very useful category, especially for other things like uh search or for code.

s102
00:10:19.960 --> 00:10:28.280
Uh you have doc corpora corpora corpora of documents, you have repos that are also very useful for kind of anchoring your um your uh your learning as well.

s103
00:10:28.280 --> 00:10:37.160
That are And so uh taking these sources, these raw materials, as places to kind of search for tasks from is a very useful way of starting to kind of create this environment out of nothing.

s104
00:10:37.160 --> 00:10:39.240
Well, it's not nothing, it's something from the real world.

s105
00:10:39.240 --> 00:10:49.320
And because you have the real world, you want to use that, the real world, your production environment, your agent traces, as the source from which you want to learn, even if you don't have supervision yet.

s106
00:10:49.320 --> 00:10:51.880
And so one thing you have to do here is get tasks.

s107
00:10:51.880 --> 00:11:05.800
And so documents are actually a pretty easy version of this where you can just sample documents, you can have models generate question-answer pairs grounded in the documents, you can verify that those answer those questions are answerable with other models that are still grounded in the documents,

s108
00:11:05.800 --> 00:11:09.240
and then the actual task at hand involves throwing away the initial search.

s109
00:11:09.240 --> 00:11:10.600
And so you kind of get to work backwards.

s110
00:11:10.600 --> 00:11:19.280
And so this general principle of like working backwards is starting from the solution or something that's close to the solution and have where your real task is like further upstream.

s111
00:11:19.280 --> 00:11:24.760
This is a very useful way of kind of having a You can verify the easy problem and then learn on the hard problem.

s112
00:11:24.760 --> 00:11:30.040
And so anything where you can move backwards like this is super useful for kind of getting supervision for free.

s113
00:11:30.040 --> 00:11:47.160
Um in code, you can have uh you can use uh real-world PRs, the diffs, the the descriptions, the test cases, uh removing different pieces of different files to be able to have models start learning over code bases because you can kind of take something that is a completed artifact and start breaking it down into smaller pieces

s114
00:11:47.160 --> 00:11:53.920
and then have replaying these pieces of getting to an end state that you know is reachable uh be a task that you train on.

s115
00:11:53.920 --> 00:12:02.680
And so, this idea of wanting to know that an end state is reachable and that you can then take steps back, throw away the solution, and then learn to find it again

s116
00:12:02.680 --> 00:12:05.480
uh can be applied more generally beyond code as well.

s117
00:12:05.480 --> 00:12:16.600
Uh and so, we talk about world simulators broadly as the the sort of thing we might want to do in messier environments which are not just uh production uh like which are not just doc search or code.

s118
00:12:16.600 --> 00:12:24.040
And so, a lot of the ones that we've been working on at Prime and Elect are related to things like tool use and web applications where we don't actually have full controllability

s119
00:12:24.040 --> 00:12:25.000
of the back-end state.

s120
00:12:25.000 --> 00:12:31.840
There are some MCP tools or CLI tools or websites or applications where we can't actually program them yet.

s121
00:12:31.840 --> 00:12:34.600
And so, what we want to do is learn to simulate them.

s122
00:12:34.600 --> 00:12:48.280
And so, we found that using combinations of uh universal back-end infrastructure and test time scaling and search and kind of iterating between the simulator and the real uh and the real, we can kind of ground in these production traces.

s123
00:12:48.280 --> 00:12:51.520
This then allows us to create really high-fidelity simulators.

s124
00:12:51.520 --> 00:13:01.680
And so, we found that these simulators are actually really great for RL because you can one, if you have production data, you can make your simulator better and better over time, but also you have full controllability

s125
00:13:01.680 --> 00:13:02.360
over the back-end.

s126
00:13:02.360 --> 00:13:06.000
And so, you can actually do this reverse engineering where you get to kind of plant the answer.

s127
00:13:06.000 --> 00:13:18.000
You can start from the end and work backwards uh and so that you you have this verifiability baked into the simulator even if you don't have it in the real-world production deployment because you don't know in advance if a task was solvable

s128
00:13:18.000 --> 00:13:19.920
at the time that you are being asked it.

s129
00:13:19.920 --> 00:13:24.720
Um and in terms of doing this, a very useful thing is scaling judges.

s130
00:13:24.720 --> 00:13:33.080
And so, a lot of times we will have a model that does something and it will make mistakes along the way and it's easier to tell what went wrong in hindsight.

s131
00:13:33.080 --> 00:13:39.520
And so, the fact that you've already seen the chain of events after and you can look backwards and say, "Okay, the model made a mistake here.

s132
00:13:39.520 --> 00:13:40.680
This thing doesn't feel quite right."

s133
00:13:40.680 --> 00:13:44.400
Or we asked seven different models and they all kind of agree this thing is wrong.

s134
00:13:44.400 --> 00:13:49.640
This is a very useful way of kind of spending compute to do search to then extract rubrics.

s135
00:13:49.640 --> 00:13:59.480
These rubric questions are are are very effective at like kind of distilling down the the search into something that we can then use to more cheaply audit and kind of also ground,

s136
00:13:59.480 --> 00:14:04.840
once we have these rubrics, as a a way of saying, "Okay, we need tasks that target these kinds of failure modes as well."

s137
00:14:04.840 --> 00:14:08.960
Um all of this is under the umbrella of scaling search with test-time compute.

s138
00:14:08.960 --> 00:14:18.560
And so, we can scale search for mining traces by if we have offline production traces, we can just look at them more and think about them more and have more models play with them.

s139
00:14:18.560 --> 00:14:19.880
We can calibrate difficulty.

s140
00:14:19.880 --> 00:14:29.040
So, RL to have the advantage gap that we mentioned needs to have a a separation between what one model will do once and what a model what a collection of rollouts will do.

s141
00:14:29.040 --> 00:14:36.000
And so, you want tasks that are not too easy, not too hard and you want to be searching for these and iterating on generating more of them.

s142
00:14:36.000 --> 00:14:42.440
And so, this is another area where you can spend compute to um refine the difficulty of your your task distributions, your task sets.

s143
00:14:42.440 --> 00:14:56.720
Um for simulators, when you're building web applications or tools that need to simulate complex behavior, you can spend compute on searching these and then you can have agents refine the implementations of them and this allows for increasingly high-fidelity environments.

s144
00:14:56.720 --> 00:15:03.480
You can do this on verification both at train time as well as offline when you're kind of creating these rubrics.

s145
00:15:03.480 --> 00:15:09.280
You can do things like red teaming with adversarial prompt optimization to to kind of explore for backdoors.

s146
00:15:09.280 --> 00:15:14.960
Then you can look for traces and spend compute mining these traces for understanding was this the reward a reward hacker?

s147
00:15:14.960 --> 00:15:16.800
Was this actually kind of in the spirit of the task?

s148
00:15:16.800 --> 00:15:30.560
And I think these things can kind of feel like reward hacking can kind of sneak up on you if you're not careful for it, but in many cases the basic simple things actually work quite well where if the reward hacks are the sorts of things where a human can look at them and be like, oh yeah, that's

s149
00:15:30.560 --> 00:15:31.680
a reward hack.

s150
00:15:31.680 --> 00:15:33.600
Judges are often not quite good at doing this as well.

s151
00:15:33.600 --> 00:15:46.960
They just don't necessarily tell if you tell the model not to do this, it won't necessarily do it in the rollout, but in hindsight you can reflect on this and spend compute to kind of especially if you are collecting these over time and you are building up your corpus of examples of reward hacks, you can understand the

s152
00:15:46.960 --> 00:15:55.360
sorts of things that go wrong and address this by kind of again spending inference compute on refining your implementation, refining your rewards.

s153
00:15:55.360 --> 00:15:56.960
As well as validating these by training.

s154
00:15:56.960 --> 00:16:03.520
And so we find that in many cases you can do a lot up front, but also also there are things that don't show up until you actually like start doing RL.

s155
00:16:03.520 --> 00:16:15.360
And so part of this is folding in training experiments themselves into the process of environment design where you can you can do small runs with individual models on like one environment

s156
00:16:15.360 --> 00:16:18.680
and and see what happens and you can understand that the behavior changes.

s157
00:16:18.680 --> 00:16:27.360
You can have metrics that log the the types of tool calls that are being done that are like judges asking questions about the the traces to understand how behavioral patterns are changing.

s158
00:16:27.360 --> 00:16:35.839
And all of these are very useful ways of kind of getting something from nothing and using compute as the thing that allows you to refine your understanding.

s159
00:16:35.839 --> 00:16:45.440
And ultimately what you want is to surface the most important pieces up to the human, the the highest level of questions about what is actually going on, what is the goal so that the

s160
00:16:45.440 --> 00:16:54.680
all this is deferring to the human for the most important pieces of like actually employing expert to say this is good, this is bad, this is what I want, this is not what I want.

s161
00:16:54.680 --> 00:16:59.200
And uh these are all the ways that we kind of gain confidence in the environments.

s162
00:16:59.200 --> 00:17:05.079
I know we're running a little short on time, so I wanted to break up a couple blogs that we've put out recently that are kind of demonstrating pieces of this.

s163
00:17:05.079 --> 00:17:21.360
We have a blog called general agent which is demonstrating this for tool use, this online loop of generating, solving, and synthesizing new tasks and gating based on this pass rate which then we train on and we see a great uplift on popular benchmarks for tool use.

s164
00:17:21.360 --> 00:17:29.920
Um additionally, beyond just RL, we found that it's quite important to think about cases where there's information in the world that RL alone will not explore.

s165
00:17:29.920 --> 00:17:35.800
And so, there's this great work Echo from some researchers that we are friends with and have been collaborating with.

s166
00:17:35.800 --> 00:17:44.800
And then we did our own kind of deep dive into this as well to look at what happens when you have an agent that is not just training with reinforcement learning, but is also getting supervised learning signal from the environment itself.

s167
00:17:44.800 --> 00:17:55.240
Which then allows the model to understand things like having a native world model of the environment, understanding what to expect because it has a likelihood model of the tokens that the environment itself will generate.

s168
00:17:55.240 --> 00:18:05.400
And these are the sorts of things that in many cases allow the model itself to kind of more adaptively navigate the world and not just like refine its skill, but like get new information into its weights over time as well.

s169
00:18:05.400 --> 00:18:16.360
Um and so all of this is in spirit of making post-training easier, making continual learning easier, giving people the ability to create agents and not worry too much about

s170
00:18:16.360 --> 00:18:21.080
having to to fuss with the research pieces and fine-tune all the the small details.

s171
00:18:21.080 --> 00:18:30.720
Today we still do, but we're kind of seeing paths forward of how we start automating this more and more by like having environments as the anchor, which we can then spend compute on refining.

s172
00:18:30.720 --> 00:18:41.120
We can kind of use compute to mine the data we have from the real world to refine the the signals where the humans are kind of just in the same way that with coding agents we're kind of going to higher levels of abstraction,

s173
00:18:41.120 --> 00:18:43.480
we can do this with environment and reward design as well.

s174
00:18:43.480 --> 00:18:56.240
Um and all of this is well allows us to ultimately close the loop where models are then able to stay within the guardrails we give them, they go find the issues in production, and then they turn these back into new tasks that can then be trained on for getting better in the real world.

s175
00:18:56.240 --> 00:19:00.400
Um we work hands-on with startup enterprises to help them train their models.

s176
00:19:00.400 --> 00:19:02.920
We are also hiring quite a lot.

s177
00:19:02.920 --> 00:19:06.640
If you want to get in touch for either of these, find me after the talk.

s178
00:19:06.640 --> 00:19:08.228
Thanks a bunch.

s179
00:19:08.228 --> 00:19:10.228
[applause]

s180
00:19:26.160 --> 00:19:28.160
Hey.