WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=iniryju23Sk

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/iniryju23Sk.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.760 --> 00:00:14.320
Uh my name's John Feminella.

s3
00:00:14.320 --> 00:00:20.080
I work on AI architecture and engineering at Two Sigma, which is a hedge fund here in New York.

s4
00:00:20.080 --> 00:00:24.840
Before I get too far into things, I have a legal disclaimer slide that I have to show you.

s5
00:00:24.840 --> 00:00:26.320
It says two things.

s6
00:00:26.320 --> 00:00:31.840
One, I promise not to sell or market or offer you any securities at any point in this talk.

s7
00:00:31.840 --> 00:00:35.920
And two, everything in this is my own opinion and not the opinion of Two Sigma.

s8
00:00:35.920 --> 00:00:36.960
All right, thanks for indulging me.

s9
00:00:36.960 --> 00:00:43.000
If I don't say those magic words, the 5G microchips implanted in me will explode.

s10
00:00:44.160 --> 00:00:48.440
So, that slide is a good example, actually, of how our industry is regulated.

s11
00:00:48.440 --> 00:01:00.360
And as a regulated company with lots of controls, I think one thing that's always interesting is to explore what happens when controls fail, because that gives you a good idea about why the controls were there in the first place.

s12
00:01:00.360 --> 00:01:05.040
And so, I'd like to tell you a story about that by showing you another legal slide.

s13
00:01:05.040 --> 00:01:12.440
After August 2020, a very interesting contract clause suddenly started showing up in a bunch of commercial loan agreements.

s14
00:01:12.440 --> 00:01:16.200
And this clause essentially tries to prevent a certain scenario from happening.

s15
00:01:16.200 --> 00:01:18.280
And it's complicated, but I'll explain it like this.

s16
00:01:18.280 --> 00:01:24.960
Let's say that you're a lemonade stand business, and you borrow money from a lender to capitalize your lemonade stand.

s17
00:01:24.960 --> 00:01:36.680
And the bank that's overseeing that transaction, that's in the middle between the borrower and the lender, sends the lender all of the money that you owe in a catastrophic

s18
00:01:36.680 --> 00:01:38.000
mistake.

s19
00:01:38.000 --> 00:01:41.920
You'd probably be mad and want the money back, but the lender might also say, "No, thanks.

s20
00:01:41.920 --> 00:01:45.560
You paid us back and we're all good, so we'd rather keep the money."

s21
00:01:45.560 --> 00:01:48.280
Something like this happened in August 2020.

s22
00:01:48.280 --> 00:01:54.360
Revlon, the cosmetics company, had borrowed money under a big 2016 term loan.

s23
00:01:54.360 --> 00:02:02.252
And that loan was split up into a bunch of pieces across 300-plus lenders, hedge funds, credit funds, CLOs, other banks.

s24
00:02:02.252 --> 00:02:05.292
And in the middle is Citibank, the administrative agent.

s25
00:02:05.292 --> 00:02:13.852
Being the agent in this kind of transaction means that you're performing the plumbing that connects the borrowers with the lenders and managing the life cycle of the loan.

s26
00:02:13.852 --> 00:02:17.412
So, in this transaction, Revlon pays City and then City pays the lenders.

s27
00:02:17.412 --> 00:02:22.572
And City's not lending its own money, it's just in the middle moving people's money around.

s28
00:02:22.572 --> 00:02:26.132
So, here's the plan for August 11th, 2020.

s29
00:02:26.132 --> 00:02:33.212
Revlon would pay about 8 million bucks of accrued interest to both groups and the lenders' principal would roll into this new facility.

s30
00:02:33.212 --> 00:02:35.572
That's an exchange of paper, not cash.

s31
00:02:35.572 --> 00:02:38.412
So, the remaining lenders' principal would stay put.

s32
00:02:38.412 --> 00:02:44.092
And the idea here is that only interest should be going out, principal should not be moving at all.

s33
00:02:44.092 --> 00:02:48.532
And the bank had a piece of internal processing software called Flexcube.

s34
00:02:48.532 --> 00:02:51.052
Isn't that a fun enterprise software name?

s35
00:02:51.052 --> 00:03:00.452
So, in Flexcube, the easiest way to do this kind of transaction is to pretend to record it like Revlon were paying off the entire loan.

s36
00:03:00.452 --> 00:03:04.132
And that triggers the correct interest payment calculations to everyone.

s37
00:03:04.132 --> 00:03:06.492
You can probably guess what's about to happen next.

s38
00:03:06.492 --> 00:03:17.212
So, you're supposed to send the principal portion to an internal wash account that you own as the bank, so that it's booked and recorded correctly, but it never leaves the bank's perimeter.

s39
00:03:17.212 --> 00:03:19.292
So, keep your eye on that wash account.

s40
00:03:19.292 --> 00:03:29.732
What was supposed to happen is that the boxes labeled uh front and fund and principal should all include the wash account number and should all have a check box checked next to them.

s41
00:03:29.732 --> 00:03:37.292
So, unfortunately, it turned out that was what needed to happen, but for various arcane reasons, uh the boxes were not checked.

s42
00:03:37.292 --> 00:03:42.132
And although Flexcube pops up a warning notification that says, "Uh-oh, money's about to leave the bank.

s43
00:03:42.132 --> 00:03:43.852
Are you sure you want to do this?"

s44
00:03:43.852 --> 00:03:44.852
That was correct.

s45
00:03:44.852 --> 00:03:48.132
Money was supposed to leave the bank because they were making the interest payments.

s46
00:03:48.132 --> 00:03:54.052
So, the three approvers who were supposed to approve that looked at it and said, "Yep, that's the right thing that's supposed to happen."

s47
00:03:54.052 --> 00:04:00.452
And although that warning notification popped up, the human operators carrying out the transaction overrode that.

s48
00:04:00.452 --> 00:04:02.892
Remember, Revlon was making that interest payment.

s49
00:04:02.892 --> 00:04:14.494
So, the principal never went to the wash account, and instead Flexcube sent the entire outstanding principal of the loan to all of the 300 lenders, about $900 million instead of $8 million.

s50
00:04:14.494 --> 00:04:14.797
So, every

s51
00:04:14.797 --> 00:04:14.934
[snorts]

s52
00:04:14.934 --> 00:04:21.254
lender got the amount that they would have been owed if Revlon had paid the loan in full, but Revlon didn't want to pay off the loan.

s53
00:04:21.254 --> 00:04:22.774
It wanted to hold onto its money.

s54
00:04:22.774 --> 00:04:30.654
That was a great deal for the lenders because remember, Revlon was in trouble at the time, so its loans weren't trading at face value, and getting all your money back on a deal

s55
00:04:30.654 --> 00:04:35.054
that you thought was only worth 60 cents on the dollar is a great deal for you.

s56
00:04:35.054 --> 00:04:39.534
So, they didn't want to give the money back even after Citi called them and said, "Please, that was a mistake.

s57
00:04:39.534 --> 00:04:40.334
Give us the money back."

s58
00:04:40.334 --> 00:04:45.174
And their argument was, "We shouldn't have to give the money back because you paid us in full, and that's how loans work.

s59
00:04:45.174 --> 00:04:52.094
You eventually pay them back, and just because you paid it back at a different time than you were supposed to doesn't mean that we should have to give it back now."

s60
00:04:52.094 --> 00:04:58.214
And so, to prevent that from happening, uh commercial loan agreements everywhere started including including a clause like that one.

s61
00:04:58.214 --> 00:05:01.534
So, what does a payment error at a bank have to do with AI agents?

s62
00:05:01.534 --> 00:05:07.574
Well, the bank had this procedure, it had these controls, and everybody did what they thought the right thing to do was.

s63
00:05:07.574 --> 00:05:09.854
That's the problem I want to talk about for the rest of this talk.

s64
00:05:09.854 --> 00:05:15.214
Every day, we're giving real agents real autonomy over systems like Flexcube.

s65
00:05:15.214 --> 00:05:23.494
And how do we prevent accidents like that one from either from happening without locking everything down or just hoping we'll get lucky that they won't happen.

s66
00:05:23.494 --> 00:05:24.574
So, here's the crux of the issue.

s67
00:05:24.574 --> 00:05:27.254
You can't really know in advance what an agent will do, right?

s68
00:05:27.254 --> 00:05:29.574
They're these non-deterministic systems.

s69
00:05:29.574 --> 00:05:35.574
Uh and that sounds scary until you remember that humans are also non-deterministic systems, and they're going to make mistakes, too.

s70
00:05:35.574 --> 00:05:45.254
And so, most firms don't choose to say, "Well, in order to reduce the risk of humans making a mistake, we're going to like lock them in rooms and prevent them from talking to each other, and they're all going to double-check everybody else's work, right?"

s71
00:05:45.254 --> 00:05:49.894
We allow them a certain amount of autonomy along with a certain amount of controls.

s72
00:05:49.894 --> 00:05:56.454
So, we give them boundaries, visibility, and we equip them with systems and controls that let them do their jobs effectively.

s73
00:05:56.454 --> 00:06:02.014
So, in this sort of model, safety and innovation are not opposite ends of a spectrum that you must trade off on.

s74
00:06:02.014 --> 00:06:04.254
There are things that you you at the same time.

s75
00:06:04.254 --> 00:06:07.135
So, I think you can have your cake and eat it, too.

s76
00:06:07.135 --> 00:06:12.775
And when we were thinking about what controls we wanted to apply to agents, we also looked at some approaches that didn't work for us.

s77
00:06:12.775 --> 00:06:20.495
We don't want to sandbox or lock down everything, cuz a sandbox with no real access to systems is not is safe, but it's not very interesting or useful.

s78
00:06:20.495 --> 00:06:34.135
If you let agents go wild and do whatever they want, that's uh, you know, maybe powerful and freeing from a cognitive perspective, but the isolation is very weak there, and that that approach is generally not suitable for highly regulated enterprises.

s79
00:06:34.135 --> 00:06:43.575
And then, if you just have policy documents that agents look at or that are part of your uh, specification for the prompts, that's also not very enforceable at runtime.

s80
00:06:43.575 --> 00:06:47.495
You can't guarantee that that will be part of the enforcement controls.

s81
00:06:47.495 --> 00:06:54.175
So, what did work for us was thinking about what mix of controls would get us both safety and innovation at the same time.

s82
00:06:54.175 --> 00:07:00.455
So, how do we provide boundaries and stop problems, but don't hinder folks from operating correctly?

s83
00:07:00.455 --> 00:07:03.415
So, we have a number of these controls, but today I'll talk about five of them.

s84
00:07:03.415 --> 00:07:13.055
And when I say controls here, what I mean are the guardrails and shared capabilities that let teams move in ways that don't hinder what they're trying to do.

s85
00:07:13.055 --> 00:07:24.015
And we want to We don't want to reach some chasm that we need to cross to to let people get their work done before uh, and then make everyone wait while we build a bridge to get there.

s86
00:07:24.015 --> 00:07:25.735
So, these controls are composable.

s87
00:07:25.735 --> 00:07:34.135
Each one does one job, and then together, they let us run agents with real autonomy at an appropriate level of safety.

s88
00:07:34.615 --> 00:07:37.855
So, the first problem is that agents were showing up everywhere.

s89
00:07:37.855 --> 00:07:40.695
We have big teams with lots of agents running.

s90
00:07:40.695 --> 00:07:41.935
We have small teams with a few.

s91
00:07:41.935 --> 00:07:45.615
We have individual people running stuff in a uncontrolled way.

s92
00:07:45.615 --> 00:07:46.735
And these come in every shape.

s93
00:07:46.735 --> 00:07:48.015
Could be random chatbots.

s94
00:07:48.015 --> 00:07:52.615
It could be MPC servers glued together in this Frankenstein hotchpotch.

s95
00:07:52.615 --> 00:07:54.215
It could be cool notebook experiments.

s96
00:07:54.215 --> 00:07:57.375
Could be really useful web tools for folks.

s97
00:07:57.375 --> 00:08:03.095
But, it's hard to answer when you look at that a few simple questions, like who owns this?

s98
00:08:03.095 --> 00:08:04.735
What tools does it have access to?

s99
00:08:04.735 --> 00:08:05.655
What should it be doing?

s100
00:08:05.655 --> 00:08:06.735
What should it is be doing?

s101
00:08:06.735 --> 00:08:12.057
If if answer those kinds of questions, if you can't govern them, then you can't really also reuse that.

s102
00:08:12.057 --> 00:08:23.457
So, you're tempted to build multiple solutions to the same problem if you don't know about what exists already, and then the firm can't also govern those in any kind of legible way.

s103
00:08:23.457 --> 00:08:32.697
So, one thing that firms sometimes do here is add some kind of gate, like don't let anything run until everything is registered and approved and it's gone through this very onerous process.

s104
00:08:32.697 --> 00:08:42.417
And that sounds safe, but if you look at what it does, the work, every idea, even some simple notebook experiment, is going to wait in line behind something that's really risky.

s105
00:08:42.417 --> 00:08:49.697
And we don't um you know, we we don't want to subject everybody who's given the diversity of agents to exactly the same process.

s106
00:08:49.697 --> 00:08:58.937
So, most people are going to give up, or they'll just get pushed to some shadow IT, less governed path that is um less visible, and then we don't get to see anything at all.

s107
00:08:58.937 --> 00:09:01.577
So, that's what motivated the first control is an agent registry.

s108
00:09:01.577 --> 00:09:03.177
You can't govern what you can't see.

s109
00:09:03.177 --> 00:09:06.276
So, the agent registry makes agents legible, and it registers,

s110
00:09:06.276 --> 00:09:06.537
[snorts]

s111
00:09:06.537 --> 00:09:12.417
retrieves, and discovers the key facts about agents that we want to know for any agent.

s112
00:09:12.417 --> 00:09:19.017
Um so, those are things like which tools it deploys, what endpoints it's going to talk to, which product does it belong to, who owns it.

s113
00:09:19.017 --> 00:09:31.937
And humans, other programs, other agents can then use the agent registry to decide what's available at the firm, whether that's useful for them, and to go use that as the source of truth for other questions.

s114
00:09:31.937 --> 00:09:33.897
So, this doesn't require extra effort on your part.

s115
00:09:33.897 --> 00:09:40.777
It just happens when you use one of the firm's approved agent building frameworks, it's automatically registered, all that information is recorded there for you.

s116
00:09:40.777 --> 00:09:44.937
And if you want to do a manual registration, you can certainly do that.

s117
00:09:44.937 --> 00:09:51.471
So, the second problem we have is that now that we've got all these agents doing all these things, we

s118
00:09:51.471 --> 00:09:51.577
[snorts]

s119
00:09:51.577 --> 00:09:54.257
quickly notice that what they're doing is really different.

s120
00:09:54.257 --> 00:09:57.857
And that means our agents carry really different risks, right?

s121
00:09:57.857 --> 00:10:06.097
So, a thing that summarizes Jira linear tickets is really different than a production deployment agent that can touch a running system.

s122
00:10:06.097 --> 00:10:11.121
And if we can use the same if we We to use the same review process to cover all of these agents, then

s123
00:10:11.121 --> 00:10:11.178
[snorts]

s124
00:10:11.178 --> 00:10:19.818
every agent is waiting in the same queue for weeks for human review, and if people are going to route route around slow processes, then we wind up in the same situation

s125
00:10:19.818 --> 00:10:21.178
that we had before.

s126
00:10:21.178 --> 00:10:27.898
So again, you might be tempted to say the answer is something like really detailed heavy manual review by all of these different constituencies.

s127
00:10:27.898 --> 00:10:37.578
Again, that feels safe, but now we've got a trivial summarizer waiting 2 months for the same sign-offs as an agent that is touching production systems.

s128
00:10:37.578 --> 00:10:49.018
And most of the good agent ideas, most of the ones that have value right away, start out as small experiments, and you don't want those people to wait in the same line as the stuff that's touching production right away.

s129
00:10:49.018 --> 00:10:55.018
So the reviewers are drowning in these like low-risk requests, and the high-risk ones have to share that attention pool.

s130
00:10:55.018 --> 00:10:58.178
So we don't want to trade away that speed if we can help it.

s131
00:10:58.178 --> 00:11:04.378
So again, because not every agent needs the same review, and because most agents are purpose-built rather than open-ended.

s132
00:11:04.378 --> 00:11:09.058
When I say open-ended here, you can imagine coding agents like Pi or Claude Code.

s133
00:11:09.058 --> 00:11:14.538
When you open them and you start typing, uh Claude Code or Pi doesn't know what you're about to ask.

s134
00:11:14.538 --> 00:11:22.698
It doesn't It doesn't circumscribe its purpose in any particular way, but most of the agents that are deployed in enterprises do have very specific purposes.

s135
00:11:22.698 --> 00:11:28.258
When you're engaging with that agent, it's to do something specific that's well known and well understood.

s136
00:11:28.258 --> 00:11:32.258
So they have a very specific reason for existing and a very circumscribed purpose.

s137
00:11:32.258 --> 00:11:42.458
And you can tell these agents apart from each other by looking at what they declare up front, their tools, um the end points they access, the controls that that are applied to them, etc.

s138
00:11:43.458 --> 00:11:54.538
So the third problem is that if you know about your agents and what they're doing, you'll quickly notice that many agents don't talk to each other, or sorry, they don't talk to just one thing or in just one capacity.

s139
00:11:54.538 --> 00:12:03.578
The more open-ended the agent is, the more it looks like a Claude Code or Pi, the more it can do a variety and a diversity of different things.

s140
00:12:03.578 --> 00:12:07.658
Call an internal API, run a tool like a web search, that sort of thing.

s141
00:12:07.658 --> 00:12:07.713
And

s142
00:12:07.713 --> 00:12:07.818
[snorts]

s143
00:12:07.818 --> 00:12:12.138
each one of those is a different opening a different door to a different kind of risk.

s144
00:12:12.138 --> 00:12:20.899
And if you give an agent an entitlement for a role account, then it can do a lot more damage than something that's just going to read a public web page, for example.

s145
00:12:20.899 --> 00:12:29.819
So, the question is not can this agent do X, but what's the granularity that we need to solve for for the agent to act successfully?

s146
00:12:29.819 --> 00:12:39.739
And again, one tempting answer is to kind of be heavy-handed here or to give every agent the same short allow list of stuff that's safe to do and then block everything else.

s147
00:12:39.739 --> 00:12:41.739
Uh that's safe, but it's also useless.

s148
00:12:41.739 --> 00:12:50.899
An agent that can't uh that can't search or can't call an API or can't do any of those things without asking somebody first can't really do the work that it was intended for.

s149
00:12:50.899 --> 00:12:56.419
And then teams either give up on those agents or they find a way around the wall, and both outcomes are costly.

s150
00:12:56.419 --> 00:13:05.579
So, a wide kind of blanket block on everything trades away the innovation that we want to preserve and avoids making a real decision about risk.

s151
00:13:05.579 --> 00:13:11.939
Um and as author John Shedd wrote, "A ship in harbor is safe, but that is not what ships are for."

s152
00:13:11.939 --> 00:13:21.899
So, the fix here is to let agents declare their purpose upfront and say what tools they're going to use or what tools they need access to.

s153
00:13:21.899 --> 00:13:38.219
Um so, in the second column here, we might say that a research summarizer is going to be allowed to use a web tool to access research, but for a release deployer agent, we're not going to allow them to do that because they have no reason to use that tool.

s154
00:13:38.219 --> 00:13:41.619
And so, if they try to ask for it, they won't get it by default.

s155
00:13:41.619 --> 00:13:44.459
Um so, we don't need to make this kind of decision for every tool.

s156
00:13:44.459 --> 00:13:46.979
The agents just have to say which tools they want.

s157
00:13:46.979 --> 00:13:47.139
Right?

s158
00:13:47.139 --> 00:13:55.699
And and you're as the agent author, you're in the best position to say, "I need these four things out of the 500 that are available."

s159
00:13:55.699 --> 00:14:07.419
So, the way we need to enforce that is that when agents run in our managed environments, they look at the registry that we talked about before to look up what policies apply to it at initialization

s160
00:14:07.419 --> 00:14:07.699
time.

s161
00:14:07.699 --> 00:14:19.541
And that policy gets translated into enforcible statements, like which endpoints are you allowed to access, which hosts are you allowed to access, uh what other types of resources are available or not available to you.

s162
00:14:19.541 --> 00:14:28.901
And in this way, nobody's writing a ticket every time they need to change the access, nobody's going to talk to IT or some other group outside of your team.

s163
00:14:28.901 --> 00:14:36.701
As long as you conform to the risk score that we talked about before, and you're in the agent registry, you're you're governed, you're approved.

s164
00:14:36.701 --> 00:14:42.581
So, this is how we keep the cost to teams really low for deploying, building, running agents.

s165
00:14:42.581 --> 00:14:46.141
And the same control architecture works in a variety of settings.

s166
00:14:46.141 --> 00:14:54.141
The next thing we noticed was that teams were building a a ton of different ways to evaluate agents, and every stack looks different, and some teams don't build anything at all.

s167
00:14:54.141 --> 00:15:06.341
Sometimes you build like this really detailed stack for how your models work, sometimes it's sometimes there's nothing, and I think in general we'd be upset if teams were not writing unit tests, right?

s168
00:15:06.341 --> 00:15:08.781
And we consider evals to be kind of the same thing.

s169
00:15:08.781 --> 00:15:20.661
If models are changing, that's like a runtime dependency that your application has, and if your agent is going to change models, it doesn't know without the evals what's going to happen to its

s170
00:15:20.661 --> 00:15:23.421
to its outputs and to the quality of those outputs.

s171
00:15:23.421 --> 00:15:37.861
So, again, one tempting answer here is let's have some kind of central evaluation methodology that waits for a full eval suite and a committee sign off, but now every like prompt tweak, every one-line change is going to sit in that same queue.

s172
00:15:37.861 --> 00:15:42.541
So, we don't want to put more decisions on more reviewers for things like that.

s173
00:15:42.541 --> 00:15:44.741
We we don't want to trade away that iteration.

s174
00:15:44.741 --> 00:15:55.981
So, again, you wouldn't ship software without unit tests, so agents need evaluations for the same reason, and we want to provide this stuff at the level of the firm that's going to be the same for everybody.

s175
00:15:55.981 --> 00:16:01.461
So, team A writing evals, team B writing evals, they're both going to have many of the same features.

s176
00:16:01.461 --> 00:16:09.861
So, we want to have one shared approach to how we do evals, and have the shared approach be something that's run by the firm and available to everybody for doing their evals.

s177
00:16:09.861 --> 00:16:16.981
So, that covers all the common parts of evals, and all you've got to do is just write what's true about your agent.

s178
00:16:17.701 --> 00:16:24.982
Uh and finally, maybe you've had the experience of being asked to approve lots and lots of individual agent actions when there's a human in the loop.

s179
00:16:24.982 --> 00:16:35.742
So, that's fine at a small scale, but once you're operating agents in parallel or in production, this becomes completely onerous and review requests come in faster than anybody can be

s180
00:16:35.742 --> 00:16:37.182
expected to approve them.

s181
00:16:37.182 --> 00:16:44.462
I mean, after the first few times of this happening, you succumb to review fatigue where you just see yes, yes, yes, yes, yes, and eventually you click yes on something that's,

s182
00:16:44.462 --> 00:16:46.782
you know, activating a new nuclear bomb, right?

s183
00:16:46.782 --> 00:16:52.062
So, you don't want to click yes by reflex, but that's going to happen inevitably.

s184
00:16:52.062 --> 00:17:01.422
And one recent example of this is, uh, earlier this year a director at an AI research lab told her open claw agent to suggest emails to delete from her inbox and then wait for her approval,

s185
00:17:01.422 --> 00:17:07.742
but her inbox was so large that eventually the agent compacted its context and lost the part that said, "Wait for my approval."

s186
00:17:07.742 --> 00:17:12.582
And then it deleted hundreds of emails and didn't stop when she told it to stop from her phone.

s187
00:17:12.582 --> 00:17:17.582
So, again, you might be tempted to try to make a human approve every action, but that's not the right way to do it.

s188
00:17:17.582 --> 00:17:30.942
A better approach is to let these independent, conservative classifiers make decisions about routine calls and then send a person only the requests that are most important, the ones that are most likely to make a difference.

s189
00:17:30.942 --> 00:17:40.422
If you're saying yes 99.99999% of the time, then that's probably just a rule that the agent should follow, not a request you need to ask the human for permission for.

s190
00:17:40.422 --> 00:17:48.262
And the stuff that's unclear or ambiguous or potentially dangerous, yes, surface that to a human or if it's definitely just dangerous, deny it outright.

s191
00:17:48.262 --> 00:17:53.342
So, this is a way to not have to do all the review work all the time.

s192
00:17:53.342 --> 00:18:00.782
So, we don't want to take the same approach for both explicit, uh, when an explicit rule is applied, we want deterministic fast paths.

s193
00:18:00.782 --> 00:18:06.582
When explicit rules don't apply and there's some ambiguity, we want a classifier to do that work instead.

s194
00:18:06.582 --> 00:18:12.462
So, putting those five pieces back together, you create a registry so that every agent is visible from the day it's created.

s195
00:18:12.462 --> 00:18:15.982
You score each agent so human review goes in the right spot.

s196
00:18:15.982 --> 00:18:19.782
You run on guardrails so that they only touch what they need.

s197
00:18:19.782 --> 00:18:24.623
You share evals and metrics so everybody is reaching the same bar without having to reinvent the wheel.

s198
00:18:24.623 --> 00:18:30.783
And then finally you govern your runtime to make sure that classification happens there and not somewhere else.

s199
00:18:30.783 --> 00:18:33.103
So each of these controls works together.

s200
00:18:33.103 --> 00:18:34.983
So I'll leave you with one question.

s201
00:18:34.983 --> 00:18:37.103
Why do cars have brakes?

s202
00:18:37.103 --> 00:18:41.503
It's not just so that you can stop, it's so that you can go fast and stop safely.

s203
00:18:41.503 --> 00:18:44.143
So controls for our agents works the same way.

s204
00:18:44.143 --> 00:18:45.503
Thank you very much.

s205
00:18:45.503 --> 00:18:47.922
I'm John Feminella and I hope you had a good time.

s206
00:18:47.922 --> 00:18:49.232
[applause]

s207
00:18:49.232 --> 00:18:51.232
[music]