WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=rbjWzZK2LU0

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/rbjWzZK2LU0.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.960 --> 00:00:14.320
Hello.

s3
00:00:14.320 --> 00:00:14.880
All right.

s4
00:00:14.880 --> 00:00:16.400
Uh, let's get started.

s5
00:00:16.400 --> 00:00:19.840
So, um, well, welcome to the talk.

s6
00:00:19.840 --> 00:00:23.359
Give the agent a budget, not a not a token.

s7
00:00:23.359 --> 00:00:24.800
Um, quick intro about me.

s8
00:00:24.800 --> 00:00:25.760
My name is Sachin.

s9
00:00:25.760 --> 00:00:28.480
Um, I'm an engineer on the CI team at Entropic.

s10
00:00:28.480 --> 00:00:33.680
Uh we basically build and manage all of the test machinery for all the code that people write.

s11
00:00:33.680 --> 00:00:40.160
Uh so everything from test quarantining to merge automation to CI autoscaling to merge cues, everything and anything in between.

s12
00:00:40.160 --> 00:00:45.040
Um it's it's basically the plumbing that allows a few thousand engineers every day to ship code safely.

s13
00:00:45.040 --> 00:00:47.280
Uh I've been at the company for a little over 10 months.

s14
00:00:47.280 --> 00:00:48.480
I'm based out of Seattle.

s15
00:00:48.480 --> 00:00:54.160
Um and outside of work, I love dialing my espressos and climbing some crazy mountains.

s16
00:00:54.160 --> 00:00:58.399
So a lot of cool agent demos they they start with the same way really.

s17
00:00:58.399 --> 00:01:04.799
So someone gives an agent a god token um and gives it an access to some sort of tool list and just watch as it goes.

s18
00:01:04.799 --> 00:01:10.080
So and it does go like in in this particular example it it'll spin up like a cute little coffee website.

s19
00:01:10.080 --> 00:01:14.720
It'll build all the pages, deploy them, uh bring it up live and it's done in 3 seconds.

s20
00:01:14.720 --> 00:01:18.960
Uh people will nod people will clap and someone will ship it into production.

s21
00:01:18.960 --> 00:01:27.115
And this this talk is basically about what happens after that after the demo ships once the agent is starting to do some like real work in production.

s22
00:01:27.115 --> 00:01:28.479
[snorts]

s23
00:01:28.479 --> 00:01:32.400
So here's what after looked like in in our one of our scenarios.

s24
00:01:32.400 --> 00:01:34.400
So what you're looking at is like a real command.

s25
00:01:34.400 --> 00:01:36.880
Um the agent was trying to clean up after itself.

s26
00:01:36.880 --> 00:01:42.320
Um it it was basically listing a bunch of workloads that were no longer interesting or useful to itself.

s27
00:01:42.320 --> 00:01:44.880
Um and then it found them and deleted them.

s28
00:01:44.880 --> 00:01:56.000
Now that's that's completely reasonable except one stage in the pipeline basically evaluated to nothing and the filter dropped out and now the selector matched everything.

s29
00:01:56.079 --> 00:02:05.040
So you can see it took out about 200 workloads uh which ended up impacting about 20 engineers worth of stuff and all of that was gone in 90 seconds.

s30
00:02:05.040 --> 00:02:11.200
Nobody was being malicious in this case like the agent genuinely thought it was tidying up after itself.

s31
00:02:11.200 --> 00:02:14.640
uh some of these workloads were long running training jobs and stuff.

s32
00:02:14.640 --> 00:02:22.000
Maybe some of these were not even checkpointed and it was just like hours of progress that was gone poof in like 90 seconds and stuff, right?

s33
00:02:22.000 --> 00:02:36.239
Um the the problem in in this case is that this idea of like here's a token and here's a tool list that is just like not enough like it just doesn't scale when the agent is starting to do some some like real work in production.

s34
00:02:37.120 --> 00:02:40.560
So here's just a brief summary of like what we're going to talk about today.

s35
00:02:40.560 --> 00:02:45.680
Uh I'm going to walk through three primitives um and asymmetric verbs.

s36
00:02:45.680 --> 00:02:48.080
We we're going to look at some like rate limits.

s37
00:02:48.080 --> 00:02:51.360
We're going to look at this idea of like trip wires over allow lists.

s38
00:02:51.360 --> 00:02:59.120
And then I'm going to talk about this one lens that which I call the undo test that you can sort of use to size up the other three primitives.

s39
00:02:59.120 --> 00:03:05.200
Uh so three things that you basically enforce and then one question you sort of ask about all of them.

s40
00:03:06.879 --> 00:03:12.239
So here's basically what bugged me uh after the cleanup was done for the incident that I was just showing you.

s41
00:03:12.239 --> 00:03:16.480
Uh the the agent technically hadn't done anything that I couldn't have done.

s42
00:03:16.480 --> 00:03:18.560
Uh it was using my token after all.

s43
00:03:18.560 --> 00:03:20.560
The the failure wasn't the model itself.

s44
00:03:20.560 --> 00:03:27.519
The failure was that I was giving the agent unbounded amount of power to do something that I wasn't watching super intently.

s45
00:03:27.519 --> 00:03:37.440
Um and it was we've basically solved this the same kind of problem except not for agents but like the first time you onboarded like a junior engineer on onto your own teams, right?

s46
00:03:37.440 --> 00:03:46.640
So just think about it like we don't we don't basically sit around watching um every engineer or new boarded person just like type out every keystroke on their keyboard, right?

s47
00:03:46.640 --> 00:03:49.680
Like we're not behind their chairs just watching everything that they're doing.

s48
00:03:49.680 --> 00:03:53.519
Uh there is always a path on whenever a new engineer wants to ask for something.

s49
00:03:53.519 --> 00:03:55.920
Um there is always an escalation path.

s50
00:03:55.920 --> 00:04:00.319
uh the catastrophic stuff is just structurally out of reach for them.

s51
00:04:00.319 --> 00:04:03.599
So the the other side of the coin is that the agents are very different.

s52
00:04:03.599 --> 00:04:04.720
They they never get tired.

s53
00:04:04.720 --> 00:04:05.680
They never sleep.

s54
00:04:05.680 --> 00:04:08.159
And every so often they're just like very confidently wrong.

s55
00:04:08.159 --> 00:04:12.959
Um and if I'm being honest, that's that's maybe most of us on the first month of our jobs anyways.

s56
00:04:12.959 --> 00:04:22.720
And so what what we're going to talk about in a little bit in is just this idea of like your onboarding checklist, but written down as policy for for agents.

s57
00:04:23.600 --> 00:04:31.680
So that and that brings me to the word in the title of my my talk here is that the the the standard fix for an incident like that is basically

s58
00:04:31.680 --> 00:04:33.600
that you narrow the token scope, right?

s59
00:04:33.600 --> 00:04:36.479
Like you just take the deletes away effectively.

s60
00:04:36.479 --> 00:04:38.880
You would technically never do that for like a new hire, right?

s61
00:04:38.880 --> 00:04:41.919
Like you would you wouldn't take the whole verb away from them.

s62
00:04:41.919 --> 00:04:44.400
And it also doesn't work for an agent either.

s63
00:04:44.400 --> 00:04:53.600
Maybe it works for about a week, maybe two, but then you eventually end up in a situation where the agent is genuinely trying to delete something that it feels like is just not in the infrastructure

s64
00:04:53.600 --> 00:04:59.520
and you will just be there sitting and pressing enter by hand all over again.

s65
00:04:59.520 --> 00:05:03.680
The the core concept with a token that I feel like is wrong is that a token is a boolean.

s66
00:05:03.680 --> 00:05:06.639
It's just a yes or no. It's a static list of scopes.

s67
00:05:06.639 --> 00:05:09.039
Um you either have it or you don't have it.

s68
00:05:09.039 --> 00:05:13.280
So if the token list is too tight, then your agent is effectively useless.

s69
00:05:13.280 --> 00:05:18.720
If the token list is too wide, then you're maybe writing writing like a postmortm.

s70
00:05:18.720 --> 00:05:20.160
A budget is a very different shape.

s71
00:05:20.160 --> 00:05:23.280
A budget is is um not just like one number.

s72
00:05:23.280 --> 00:05:26.320
It just has like four different four different dimensions, right?

s73
00:05:26.320 --> 00:05:27.919
How much can the agent do?

s74
00:05:27.919 --> 00:05:29.840
Uh how fast can it do it?

s75
00:05:29.840 --> 00:05:31.360
What can it undo on its own?

s76
00:05:31.360 --> 00:05:34.960
And then who's noticing while it's actually taking those actions?

s77
00:05:34.960 --> 00:05:37.440
And that's roughly the umbrella for like my four primitives.

s78
00:05:37.440 --> 00:05:42.960
Uh is that each of the things that follow, they're basically being um one of the things along these dimensions.

s79
00:05:42.960 --> 00:05:47.680
and they're going to replace a yes or no question with a budget.

s80
00:05:47.680 --> 00:05:51.440
So, I'm going to start off with this first verb which I call asymmetric verbs.

s81
00:05:51.440 --> 00:05:55.440
Uh, and by verbs I simply means operations or actions that an agent can take.

s82
00:05:55.440 --> 00:05:59.919
So, like it could be API calls, it could be uh CLI commands, it could be really anything.

s83
00:05:59.919 --> 00:06:09.680
And they're asymmetric because the same sized action even though it looks same size, they're they can have very different blast radius in actuality depending on which direction it goes.

s84
00:06:09.680 --> 00:06:14.960
So the the core point is that you need to stop thinking about resources for a second and think about verbs.

s85
00:06:14.960 --> 00:06:20.160
So specifically think about what happens when one of the verbs goes wrong.

s86
00:06:20.160 --> 00:06:22.000
So some verbs they fail out loud.

s87
00:06:22.000 --> 00:06:31.759
So in this particular example, let's say if an agent decides to unskip a test and say it's the wrong call, the worst that would happen is CI would go red uh for a bunch of people.

s88
00:06:31.759 --> 00:06:32.960
Um same with paging.

s89
00:06:32.960 --> 00:06:38.720
Like if an agent decides to page a human and if it's the wrong call, the worst that's happening is that it's a nuisance for the on call.

s90
00:06:38.720 --> 00:06:41.280
But there's always a human to correct it.

s91
00:06:41.280 --> 00:06:43.520
There are other verbs that fail silently, right?

s92
00:06:43.520 --> 00:06:48.639
So if the agent decides to skip a test uh due to whatever reason, it shouldn't have.

s93
00:06:48.639 --> 00:06:50.479
Nothing technically turns red.

s94
00:06:50.479 --> 00:06:55.759
A real bug can actually walk into production with green checks and nobody would notice it until much later.

s95
00:06:55.759 --> 00:07:04.479
So unskip and skip in this example, they're effectively the they're the same kind of action, but the difference is which of the failures would show up on a dashboard and which one wouldn't.

s96
00:07:04.479 --> 00:07:14.319
And so the core idea is that you give um access to verbs that can fail out loud loudly on a dashboard to your agent and for the other ones just involve a human.

s97
00:07:14.319 --> 00:07:23.520
Uh just a bit of context on CI stuff and how this plays out for us is that we have a test quarantining service behind the scenes and it basically holds a list of

s98
00:07:23.520 --> 00:07:31.199
um all the tests that are currently skipped because an on call decided that it had to uh they had to like break glass a certain situation whenever there was like an incident or something.

s99
00:07:31.199 --> 00:07:39.680
Now the agent has the ability to reenable any one of these own depending on like when it evaluates when the tests are uh working fine and stuff.

s100
00:07:39.680 --> 00:07:47.599
Again if it's the wrong call the worst that would happen is a bunch of different builds and tests would just like start showing up red and then a human can actually put them back very cheaply.

s101
00:07:47.599 --> 00:07:55.680
The skip is a break glass verb itself right like it's basically as I mentioned what our on call would reach for uh during a very critical situation during under pressure and stuff.

s102
00:07:55.680 --> 00:08:03.120
And so in that scenario if like uh an agent actually gets it wrong a real production bug can actually a real bug can walk into production.

s103
00:08:03.120 --> 00:08:06.560
So this needs a human and it will always leave an audit trail.

s104
00:08:06.560 --> 00:08:11.599
Uh and the key detail is that the agent itself is not responsible for writing the row or the audit trail itself.

s105
00:08:11.599 --> 00:08:14.800
There is a proxy in the middle that I've highlighted that I'm going to talk about in a little bit.

s106
00:08:14.800 --> 00:08:20.800
Uh which is responsible for stamping the caller's main identity on every call whether it's a skip or an unskip.

s107
00:08:20.800 --> 00:08:26.479
um the agent technically never holds uh the pen on its own provenence.

s108
00:08:26.960 --> 00:08:28.720
The second primitive is about rate limits.

s109
00:08:28.720 --> 00:08:33.919
Uh it's pretty standard concept but and this is the most concrete form of like the budget idea as a whole.

s110
00:08:33.919 --> 00:08:35.919
So a ceiling that refills.

s111
00:08:35.919 --> 00:08:40.640
So every caller gets a small amount of disruptive actions uh per time window.

s112
00:08:40.640 --> 00:08:42.479
So you can spend them however you want.

s113
00:08:42.479 --> 00:08:44.399
Uh there's no approval, there's no waiting.

s114
00:08:44.399 --> 00:08:50.399
And if you cross the line, the request simply bounces back with a count saying that you're actually exceeding your count um your budget.

s115
00:08:50.399 --> 00:08:52.959
You wait a bit and then the limit essentially refills.

s116
00:08:52.959 --> 00:08:54.160
And that's that's the whole thing.

s117
00:08:54.160 --> 00:08:57.440
So the agent gets full autonomy within the limit.

s118
00:08:57.440 --> 00:09:00.560
And there is a hard ceiling on how bad a single loop can get.

s119
00:09:00.560 --> 00:09:03.360
And every write effectively gets a rate limit.

s120
00:09:03.360 --> 00:09:04.959
There are no exceptions to that.

s121
00:09:04.959 --> 00:09:06.640
What changes is the size of the rate limit.

s122
00:09:06.640 --> 00:09:18.080
So if if if I'm if I'm trying to delete a bunch of workloads in my own name space like my my rate limit might be higher but if I'm trying to touch resources in a shared namespace my budget or my rate limits might be smaller.

s123
00:09:19.440 --> 00:09:23.920
So this is uh effectively the solutioning for the incident that I was showing earlier.

s124
00:09:23.920 --> 00:09:35.600
Um after the incident the team sort of that sits next to mine they they built an admission web hook of sorts uh whose sole job is to cap the number of deletes at a fixed number per hour per resource kind per name space.

s125
00:09:35.600 --> 00:09:42.320
There is always a bypass flag because sometimes you genuinely want to delete more than you're allowed for and maybe some on call scenario or whatever.

s126
00:09:42.320 --> 00:09:50.959
And the part that I absolutely love in this case is that inside a cloud code session or inside an agent session effectively the the bypass flag simply refuses to do anything.

s127
00:09:50.959 --> 00:09:55.279
All it's going to do is tell the agent to ask the human to run the command itself.

s128
00:09:55.279 --> 00:10:04.800
So the agent effectively gets the rate limit and the human keeps the override and nobody effectively has to file a ticket for uh the limit because it just like refills.

s129
00:10:05.680 --> 00:10:10.160
The third primitive is about this concept of trip wires over allow lists.

s130
00:10:10.160 --> 00:10:18.320
So in my mind like an allow list is effectively a guess that you're making up front about what the agent needs or about model behavior or agent behavior itself.

s131
00:10:18.320 --> 00:10:24.959
uh I feel like it's pretty static and you write it up front before before you have any data on like how the agent is behaving in different situations.

s132
00:10:24.959 --> 00:10:29.120
A trip wire on the other hand is how you get that data like after the fact.

s133
00:10:29.120 --> 00:10:39.760
So for cheap actions you let the agent act and every action gets recorded with the actor sort of stamp identity stamp and these two kind of go well together like rate limits are the enforcement

s134
00:10:39.760 --> 00:10:42.800
um they they put a hard limit on uh the rate itself.

s135
00:10:42.800 --> 00:10:47.760
Trip wires are how you find out what actually happened so you've got something to react to.

s136
00:10:47.760 --> 00:10:51.360
So you effectively watch the aggregate and not like individual calls.

s137
00:10:51.360 --> 00:10:57.360
And usually when a trip wire goes off the fix is like maybe one or two lines in the agents context and not really a big code change.

s138
00:10:57.360 --> 00:11:00.320
Uh the core point is that allow lists don't really get better over time.

s139
00:11:00.320 --> 00:11:04.480
They can get stale but trip wires do get better over time.

s140
00:11:04.480 --> 00:11:06.800
So here's like a loopin practice for us.

s141
00:11:06.800 --> 00:11:13.519
So like we track one number uh which is the number of investigation threads that our agent is launching per hour for a given test job failure.

s142
00:11:13.519 --> 00:11:22.160
So one morning say the number was way above the baseline and the trip fire page is on call and that part's important because a trip wire that nobody sees is practically useless.

s143
00:11:22.160 --> 00:11:27.040
Uh it pages after the right has already happened after the the limit has been crossed not before.

s144
00:11:27.040 --> 00:11:30.399
It's effectively the smoke detector not the lock on the door.

s145
00:11:30.399 --> 00:11:38.880
Uh the agent has spun up in this particular scenario a bunch of like investigation threads for dozens of job that were all failing with the same kind of error signature.

s146
00:11:38.880 --> 00:11:48.320
Each thread effectively looked reasonable on its own, but like if you took them in aggregate, you would realize that it was actually an infrastructure failure that was causing the same test failure signatures across the board.

s147
00:11:48.320 --> 00:11:59.519
So the fix in this case was as simply as telling the agent on like, hey, the next time you encounter something like this, maybe try to correlate a bunch of different failures and test jobs and stuff before launching a separate investigation thread.

s148
00:11:59.519 --> 00:12:01.200
So that correlation is important.

s149
00:12:01.200 --> 00:12:12.320
Um and the next time when this happened, uh it did exactly that because that that example and that line of like how it should react and like how it should like uh debug things was was right there to guide it.

s150
00:12:12.880 --> 00:12:14.720
So that's three rough primitives.

s151
00:12:14.720 --> 00:12:18.000
Now the lens I mentioned at the start is this idea of undo test.

s152
00:12:18.000 --> 00:12:19.200
And this one's slightly different.

s153
00:12:19.200 --> 00:12:21.839
It's it's not something you effectively enforce in code.

s154
00:12:21.839 --> 00:12:24.560
It's the question you ask when you're sizing the other three.

s155
00:12:24.560 --> 00:12:26.160
So it's two questions really.

s156
00:12:26.160 --> 00:12:28.880
One, can the agent put it back by itself?

s157
00:12:28.880 --> 00:12:32.399
and how bad would the impact be if it actually got it wrong.

s158
00:12:32.399 --> 00:12:43.600
And this this this sounds like asymmetric verbs in a sense, but it's kind of difference because the verbs ask whether you would notice the failure and undo asks whether you can recover from it.

s159
00:12:43.600 --> 00:12:51.680
So if if if the agent can effectively roll back its own change and the blast radius is acceptable to you, you effectively log it and you let it go.

s160
00:12:51.680 --> 00:12:58.639
If either of the answers is no, then you effectively need a second key and the second key is not something that the agent holds itself.

s161
00:12:58.639 --> 00:13:00.240
It has to be someone else.

s162
00:13:00.240 --> 00:13:07.519
Um, and there has to be an audit record so that you can you can track on like what happened, why the second key was involved and stuff like that.

s163
00:13:07.760 --> 00:13:12.160
Let me let me just show you this another example of like how this works outside of like the CI domain specifically.

s164
00:13:12.160 --> 00:13:16.079
So our agent has a key for one of our feature flag services.

s165
00:13:16.079 --> 00:13:21.760
So on the canary side which is basically our staging traffic and a bunch of like dog fooding customers the agent effectively has the full dial.

s166
00:13:21.760 --> 00:13:35.440
It has the ability to um ramp up a specific feature flag roll it out to uh all of the canary traffic and it can take it all the way from zero to 100 and it also has the ability to sort of uh look at any bugs that are being filed and toggle it back off and on.

s167
00:13:35.440 --> 00:13:39.519
What the agents key is not scoped to do is for promoting the flag to real production.

s168
00:13:39.519 --> 00:13:48.320
The best that an agent can do for now in a lot of cases is that it can propose uh that an that someone actually promote the feature flag in production because it's been tested out in Canary.

s169
00:13:48.320 --> 00:13:49.839
But that's pretty much it.

s170
00:13:49.839 --> 00:13:53.920
Um the second key in this scenario is not necessarily a new O system.

s171
00:13:53.920 --> 00:13:58.800
It's a scoped key for production and a scoped key for Canary.

s172
00:13:59.839 --> 00:14:02.880
So this is what it looks like day-to-day for for me personally.

s173
00:14:02.880 --> 00:14:09.199
is like I I ask cla tag in our slack channel to to own the loop the entire loop uh on on rolling out a flag.

s174
00:14:09.199 --> 00:14:14.560
It can it can basically ask me a clarifying question on whether this is for internal dog fooding group or early access or whatever.

s175
00:14:14.560 --> 00:14:18.399
Uh and the important part is that I'm not in the middle of any of these things.

s176
00:14:18.399 --> 00:14:22.240
Every action that the agent is taking is stamped with its own identity and not mine.

s177
00:14:22.240 --> 00:14:27.040
And that stamp is basically what ties the whole thing the whole thing together.

s178
00:14:27.120 --> 00:14:37.360
So the these these primitives they compose together nicely because each one asks us a different question right the verbs ask what the agent is touching the rate limit asks how often

s179
00:14:37.360 --> 00:14:51.440
and the trip wires catch what happened afterwards and the undo test is sort of the length that I personally used to to size all these three now the original sort of cold open delete that I was showing you the the incident itself the rate limit itself would have capped it at a few couple of tens of workloads

s180
00:14:51.440 --> 00:14:58.000
itself the undo test if you would be thinking about it it will basically tell you that you can't unddelete a running job in someone else's namespace.

s181
00:14:58.000 --> 00:15:03.440
Um so so anything past the cap basically need a human with the with a with a second key.

s182
00:15:03.440 --> 00:15:07.440
So the the important part is that you don't need all the checks in every right scenario.

s183
00:15:07.440 --> 00:15:11.440
Only some of these might be relevant for like whatever kind of action you're trying to evaluate.

s184
00:15:11.440 --> 00:15:22.880
Uh and if any of these really sound familiar, it it should because this is effectively some sort of onboarding checklist for your engineers like what can a new engineer touch, how much rope do they get, who signs off on their operations,

s185
00:15:22.880 --> 00:15:25.040
and how do we know it's effectively working?

s186
00:15:25.040 --> 00:15:26.399
We just wrote it for people.

s187
00:15:26.399 --> 00:15:30.240
It's now the same checklist that we want for agents.

s188
00:15:30.959 --> 00:15:33.600
So where does all the policy that I've been talking about really live?

s189
00:15:33.600 --> 00:15:36.079
Uh it's two places and I feel like you need both.

s190
00:15:36.079 --> 00:15:37.519
The first one is text.

s191
00:15:37.519 --> 00:15:38.639
Uh that's prompts.

s192
00:15:38.639 --> 00:15:42.399
That's your context files markdown the agent reads before it can act.

s193
00:15:42.399 --> 00:15:44.959
And this is where you can explain your why the the intent.

s194
00:15:44.959 --> 00:15:49.839
We've we got exactly the same sentence that I was showing earlier um written down in a markdown file.

s195
00:15:49.839 --> 00:15:52.240
It works about 80% of the time.

s196
00:15:52.240 --> 00:15:56.480
Uh the the upside is that it's very cheap to change and you can explain the reasoning.

s197
00:15:56.480 --> 00:16:02.079
The downside is you have to garden it because the the files can grow over time and at the end of the day it's just advice.

s198
00:16:02.079 --> 00:16:06.320
Uh text can shape the intent but there is no enforcement anywhere.

s199
00:16:06.320 --> 00:16:10.399
The the second place is infrastructure uh which is the proxy layer for us.

s200
00:16:10.399 --> 00:16:12.639
So the proxy is is not reading the prompt.

s201
00:16:12.639 --> 00:16:16.639
It doesn't know why the agent wants to do something and it doesn't really care.

s202
00:16:16.639 --> 00:16:18.079
It will see a delete happening.

s203
00:16:18.079 --> 00:16:22.880
It'll see like a budget being crossed and it'll simply return a 403 and that's the whole conversation really.

s204
00:16:22.880 --> 00:16:23.519
It's narrow.

s205
00:16:23.519 --> 00:16:24.959
It's deterministic.

s206
00:16:24.959 --> 00:16:28.560
It it counts, compares, it can allow a delete or deny.

s207
00:16:28.560 --> 00:16:34.160
What it can't do is explain the why and a clever like prompt injection cannot really talk it out of the rule itself.

s208
00:16:34.160 --> 00:16:35.680
So you need both.

s209
00:16:35.680 --> 00:16:42.320
Uh the text shapes what an agent is trying to do and infra is bounding how wrong can it go.

s210
00:16:42.959 --> 00:16:47.199
So zooming out, this is where uh the the infrastructure sort of really lives for us.

s211
00:16:47.199 --> 00:16:50.800
So like every agent session has its own proxy running right next to it.

s212
00:16:50.800 --> 00:16:53.279
The agent starts off by reading its own context file.

s213
00:16:53.279 --> 00:17:00.800
The the markdown files that I was showing earlier that's that's the text layer and every outbound call goes through the proxy after that point which is the infrastructure layer.

s214
00:17:00.800 --> 00:17:08.319
So the proxy is not like whatever the agent is trying to do the the proxy is only responsible for stamping that action with the agent's identity.

s215
00:17:08.319 --> 00:17:13.760
Uh like if the if the if the agent is trying to launch a bunch of jobs in a Kubernetes cluster the the proxy is not following it.

s216
00:17:13.760 --> 00:17:17.439
what is following is the stamp itself, the identity itself.

s217
00:17:17.439 --> 00:17:21.919
So the in this example, the cluster would write the stamp onto the job as a label.

s218
00:17:21.919 --> 00:17:33.679
Um and every child job or anything that's happening afterwards simply inherits the same identity and the every safeguard that is there in the rest of our systems that that are simply they're simply reading that like one label.

s219
00:17:33.679 --> 00:17:43.280
Uh whether it's ownership, whether it's kotaas, rate limits, approvals, trip wires, whatever it is, they're all keyed on the same stamp and the agent never got to touch it.

s220
00:17:43.679 --> 00:17:45.520
So that's roughly the shape of it.

s221
00:17:45.520 --> 00:17:53.600
But before I wrap up, I want to spend like 30 seconds on why it has to be a proxy layer, like a different layer that is doing the stamping on this identity and not the caller itself.

s222
00:17:53.600 --> 00:17:56.400
Because without it, the agent can roughly pick a different name.

s223
00:17:56.400 --> 00:18:00.559
Um, and every limit that you've basically set resets.

s224
00:18:01.120 --> 00:18:05.600
So say the agent has the ability to set its own identity in a header, right?

s225
00:18:05.600 --> 00:18:11.520
So let's say it hits a specific limit and what's the easiest fix from an agent's point of view that it can do?

s226
00:18:11.520 --> 00:18:12.960
it'll just change the header.

s227
00:18:12.960 --> 00:18:19.039
In this case, it'll just say instead of such in such in two and voila, you just have a fresh budget to work with.

s228
00:18:19.039 --> 00:18:21.760
Now, in this case, you technically don't have a rate limit.

s229
00:18:21.760 --> 00:18:23.280
You just have a suggestion.

s230
00:18:23.280 --> 00:18:26.320
Uh with the proxy in the path, the agent never gets to say who it is.

s231
00:18:26.320 --> 00:18:27.840
The proxy already knows.

s232
00:18:27.840 --> 00:18:35.120
It's the thing that's holding real credentials and it stamps every call with the identity that it already knows, not the one that agent claims.

s233
00:18:35.120 --> 00:18:48.320
And because proxy is the one that is stamping, you also get this per session ID so that you're able to differentiate different sessions that are running all for you and see which one's overreacting or which one's not acting as it's supposed to be.

s234
00:18:49.039 --> 00:18:59.760
U so again just stuffs that I want you to take home with is give the agent access to verbs that would fail out loud uh and keep the human on the ones that can fail out quietly.

s235
00:18:59.760 --> 00:19:06.080
Two, you you put a ceiling on every right and that has the ability to refill on its own so nobody's effectively filing tickets.

s236
00:19:06.080 --> 00:19:11.520
Three, you watch the aggregate and not the individual calls that help you understand the agents behavior.

s237
00:19:11.520 --> 00:19:14.480
Um, and then you fix whatever you can with the sentence.

s238
00:19:14.480 --> 00:19:21.760
And four, you use the undo test to sort of size all of these three or any other u primitive that you have for your own write operations.

s239
00:19:21.760 --> 00:19:28.480
And underneath all of this uh is is this concept of identity which has to come from the infrastructure, not from the request.

s240
00:19:28.480 --> 00:19:29.679
I feel like that's one rule.

s241
00:19:29.679 --> 00:19:34.000
If you get that one rule right, everything else is uh just just tuning.

s242
00:19:34.000 --> 00:19:35.738
Thank you.

s243
00:19:35.738 --> 00:19:37.738
[applause]

s244
00:19:50.148 --> 00:19:52.148
[music]