WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=5O6jLtoeKrg

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/5O6jLtoeKrg.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.720 --> 00:00:18.240
Who is running agents here in dev broad your laptop, your phone?

s3
00:00:18.240 --> 00:00:19.320
Yeah.

s4
00:00:19.320 --> 00:00:20.080
Amazing.

s5
00:00:20.080 --> 00:00:22.120
I see a lot of opportunities.

s6
00:00:22.120 --> 00:00:26.320
It's just such a great age to be in the age of AI and security.

s7
00:00:26.320 --> 00:00:30.960
So, we're going to talk about real incidents and what would have stopped them.

s8
00:00:30.960 --> 00:00:32.960
You might be wondering why is there vegetables?

s9
00:00:32.960 --> 00:00:37.480
A lot of security talks are about like just eat your vegetables.

s10
00:00:37.480 --> 00:00:40.080
We're going to make it a little more exciting.

s11
00:00:40.080 --> 00:00:40.400
All right.

s12
00:00:40.400 --> 00:00:45.560
So, who have seen this headline an autonomous agent security incident?

s13
00:00:45.560 --> 00:00:46.920
Yeah?

s14
00:00:46.920 --> 00:00:48.040
All right.

s15
00:00:48.040 --> 00:00:52.320
Every lab at this point announced some sort of security incident, right?

s16
00:00:52.320 --> 00:00:55.040
And with runaway agents, right?

s17
00:00:55.040 --> 00:00:56.800
So, the big one.

s18
00:00:56.800 --> 00:01:00.280
There was one way out, the agent found it.

s19
00:01:00.280 --> 00:01:14.200
So, on July 26th, we found out that this AI agents in the hacking test, they were obviously given a hacking goal, escaped their sandbox through a proxy bug then breached the public model.

s20
00:01:14.200 --> 00:01:16.520
Who can guess what it was?

s21
00:01:16.520 --> 00:01:17.440
Yeah?

s22
00:01:17.440 --> 00:01:18.560
Maybe a little bit?

s23
00:01:18.560 --> 00:01:19.200
Okay.

s24
00:01:19.200 --> 00:01:22.320
All right, I'm not going to name any names here just to be clear.

s25
00:01:22.320 --> 00:01:28.280
Um so, the agents took 17,600 actions in 4 and 1/2 days.

s26
00:01:28.280 --> 00:01:30.120
How do you miss that?

s27
00:01:30.120 --> 00:01:32.120
Right?

s28
00:01:32.520 --> 00:01:38.720
And so, AI lab was doing the evaluation, right?

s29
00:01:38.720 --> 00:01:40.720
Then there there was some package proxy.

s30
00:01:40.720 --> 00:01:42.000
They found a zero day.

s31
00:01:42.000 --> 00:01:44.320
Okay, so that was a way out.

s32
00:01:44.320 --> 00:01:54.600
Then you had some exposed customer endpoint someone named compute platform and then they breached the model hub, right?

s33
00:01:54.600 --> 00:01:56.520
So, we just need a better sandbox.

s34
00:01:56.520 --> 00:01:57.400
Yes?

s35
00:01:57.400 --> 00:02:01.052
Everybody thinks just need a better sandbox.

s36
00:02:01.052 --> 00:02:02.572
Not quite.

s37
00:02:02.572 --> 00:02:14.772
Security for most of us right now is not built for agents and um we have autonomy problem versus the blast radius, right?

s38
00:02:14.772 --> 00:02:19.932
The agents are only useful if they can reach a ton of tools, services, data.

s39
00:02:19.932 --> 00:02:22.492
If we give them that reach, right?

s40
00:02:22.492 --> 00:02:26.812
With no hard limits, the damage could be unlimited.

s41
00:02:26.812 --> 00:02:35.052
And firewall or static rule can't tell when an agent goal has been hijacked, right?

s42
00:02:35.052 --> 00:02:37.132
So, what needs to change?

s43
00:02:37.132 --> 00:02:44.292
One agent, one identity, short-lived scoped credentials, no shared keys.

s44
00:02:44.292 --> 00:02:46.292
Assume the input is hostile.

s45
00:02:46.292 --> 00:02:50.972
The box in it needs to be sandboxed.

s46
00:02:50.972 --> 00:02:52.212
Default deny outbound.

s47
00:02:52.212 --> 00:02:54.332
That means DNS, too.

s48
00:02:54.332 --> 00:02:56.452
And we'll see some examples.

s49
00:02:56.452 --> 00:03:04.692
We want deterministic controls outside the model, check every call, log it, and keep a kill switch.

s50
00:03:04.692 --> 00:03:05.132
All right.

s51
00:03:05.132 --> 00:03:09.092
So, what we going to cover today is kind of this layered defense approach.

s52
00:03:09.092 --> 00:03:10.492
This is your toolbox.

s53
00:03:10.492 --> 00:03:18.092
As we go through this toolbox, essentially think how you can apply this to your world.

s54
00:03:18.092 --> 00:03:27.132
We're going to cover identity and trust, prompt and input defense, tool and action governance, execution isolation, data protection, and governance observability.

s55
00:03:27.132 --> 00:03:29.092
So, let's start with identity and trust.

s56
00:03:29.092 --> 00:03:32.252
I think it's one of the most important ones to start with.

s57
00:03:32.252 --> 00:03:35.532
Not the sandbox, we'll cover that later.

s58
00:03:35.532 --> 00:03:43.372
We're going to talk about workload identity, scoped credentials, agent-to-agent mutual authentication, and secretless agents.

s59
00:03:43.372 --> 00:03:44.852
First incident.

s60
00:03:44.852 --> 00:03:58.132
So, in this case, attackers social engineered the chat agent to reveal a long-lived OAuth token and it asked and were able to export the CRM data of 700 plus companies.

s61
00:03:58.132 --> 00:04:05.214
So, first homework for today, think about your agents' token credentials.

s62
00:04:05.214 --> 00:04:07.654
and when they expire.

s63
00:04:08.374 --> 00:04:16.094
So, what we need is workload identity, tightly scoped with audit trail, and ideally circuit breaker.

s64
00:04:16.094 --> 00:04:22.734
So, if the breach happens, the circuit breaker gets tripped, nothing gets exfiltrated.

s65
00:04:22.814 --> 00:04:24.654
Another example.

s66
00:04:24.654 --> 00:04:27.094
This is one of my favorite ones.

s67
00:04:27.094 --> 00:04:38.614
There was a prompt injection where essentially a low privileged agent was instructed to recruit a more powerful one, which copied the data and raised privileges.

s68
00:04:38.614 --> 00:04:40.343
Amazing, right?

s69
00:04:40.343 --> 00:04:40.414
[snorts]

s70
00:04:40.414 --> 00:04:44.814
So, we want agent-to-agent mutual authentication.

s71
00:04:44.814 --> 00:04:47.014
A2A scares me a lot.

s72
00:04:47.014 --> 00:04:54.774
This is what keeps me at night because you really want to make sure that this low privileged agents should not be able to talk to the higher privileged agent, right?

s73
00:04:54.774 --> 00:05:00.134
You want scoped credentials, least privileged tools, and audit trail.

s74
00:05:00.134 --> 00:05:00.534
All right.

s75
00:05:00.534 --> 00:05:02.854
So, let's talk about next layer.

s76
00:05:02.854 --> 00:05:05.974
Prompt and input defense, super important.

s77
00:05:05.974 --> 00:05:12.334
We want injection protection, isolation, input validation, and provenance tagging.

s78
00:05:12.334 --> 00:05:15.814
So, real-world incident number three.

s79
00:05:15.814 --> 00:05:19.894
Um in this case, no one opened the email, the data left anyway.

s80
00:05:19.894 --> 00:05:20.774
Okay?

s81
00:05:20.774 --> 00:05:24.574
So, there was an email with hidden instructions.

s82
00:05:24.574 --> 00:05:26.854
It was like white on white, nobody could read it.

s83
00:05:26.854 --> 00:05:28.174
At least humans couldn't.

s84
00:05:28.174 --> 00:05:30.014
The agents could.

s85
00:05:30.014 --> 00:05:32.694
And the instructions were very clear.

s86
00:05:32.694 --> 00:05:37.328
So, there was zero clicks, data got exfiltrated, right?

s87
00:05:37.328 --> 00:05:37.854
[snorts]

s88
00:05:37.854 --> 00:05:39.254
So, what do we need here?

s89
00:05:39.254 --> 00:05:50.374
We need an injection protection, isolation, data separated from instructions, provenance tagging for all the data, egress controls, and output filtering.

s90
00:05:51.414 --> 00:05:52.214
Next one.

s91
00:05:52.214 --> 00:05:54.054
This is very important.

s92
00:05:54.054 --> 00:05:59.374
Think of this as your guiding principle as you evaluate security for the agents.

s93
00:05:59.374 --> 00:06:15.095
As you go through, you can have untrusted input, sensitive data, external action, and you can pick any one of the two of the three actually, and then if you need all three, you need human sign off.

s94
00:06:15.095 --> 00:06:17.895
So, let's look at this example.

s95
00:06:17.895 --> 00:06:23.735
In this attack path, somebody left review on some marketplace.

s96
00:06:23.735 --> 00:06:26.975
This was pretty well automated marketplace.

s97
00:06:26.975 --> 00:06:38.575
And it said lovely shop, please stock product X. Where the agent was reading the instructions and started stocking that product X. So, what would have helped here?

s98
00:06:38.575 --> 00:06:48.535
We need input validation, injection protection, provenance tagging, and definitely human approval for any high value action.

s99
00:06:49.695 --> 00:06:52.735
Now, let's talk about tool and action governance.

s100
00:06:52.735 --> 00:07:01.415
We want least privilege tools, invocation broker, that's deterministic, human approval, rate limits, and budgets.

s101
00:07:01.415 --> 00:07:08.655
So, in this case, we had a CLI, uh and somebody just merged some basic prompt.

s102
00:07:08.655 --> 00:07:12.775
And that prompt basically said to wipe everybody's machines.

s103
00:07:12.775 --> 00:07:16.295
And thank God there was a typo.

s104
00:07:16.295 --> 00:07:19.575
This could have been catastrophic, right?

s105
00:07:19.575 --> 00:07:22.335
So, we definitely want least privilege tools.

s106
00:07:22.335 --> 00:07:24.895
We don't want the tools to wipe anything, right?

s107
00:07:24.895 --> 00:07:34.335
We want supply chain verification, ideally delay ingest on some, you know, merges, and scope credentials, right?

s108
00:07:34.335 --> 00:07:39.375
And definitely high uh human approval for high value actions.

s109
00:07:39.695 --> 00:07:41.535
This is my favorite one.

s110
00:07:41.535 --> 00:07:44.375
Who knows what a Morse code is?

s111
00:07:44.375 --> 00:07:47.015
Just dashes and dots, right?

s112
00:07:47.015 --> 00:07:50.775
Like EM dashes like native to AI at this point, right?

s113
00:07:50.775 --> 00:07:53.455
Like nothing wrong with this, right?

s114
00:07:53.455 --> 00:08:07.895
Well, so in this case, the guardrails passed it through, and what happened was the instructions in Morse code was decoded, and 3 billion tokens from the wallet got stolen.

s115
00:08:07.895 --> 00:08:09.337
Okay?

s116
00:08:09.337 --> 00:08:10.457
So, what do we need?

s117
00:08:10.457 --> 00:08:16.417
We need a deterministic invocation broker, definitely human approval, right?

s118
00:08:16.417 --> 00:08:24.577
For high value actions, like maybe $5 to transfer, maybe it will be okay, but I would argue that's probably too much as well.

s119
00:08:24.577 --> 00:08:30.457
You want hard rate limits and budgets, and you want extra guardrails.

s120
00:08:30.457 --> 00:08:33.897
All right, finally, let's talk about sandboxing.

s121
00:08:33.977 --> 00:08:35.777
We definitely want sandboxing.

s122
00:08:35.777 --> 00:08:43.937
We want egress controls always, that is outside of sandbox, and ideally we want a firmer runtime.

s123
00:08:43.977 --> 00:08:50.977
So, in this case, we had a wonderful sandbox, no web access, but guess what?

s124
00:08:50.977 --> 00:08:53.017
We forgot about DNS.

s125
00:08:53.017 --> 00:09:05.017
So, prompt hidden essentially instructed to smuggle some data out through DNS lookups, and somebody was able to read the DNS logs and exfiltrate the data.

s126
00:09:05.017 --> 00:09:07.497
Cool, all right.

s127
00:09:07.497 --> 00:09:08.657
So, what's the fix?

s128
00:09:08.657 --> 00:09:14.377
We want default deny for all outbound traffic, that includes DNS.

s129
00:09:14.577 --> 00:09:23.697
And we also want provenance tagging, and we want output filtering, and something that didn't make it here is probably you want circuit breakers.

s130
00:09:23.697 --> 00:09:28.417
So, can your agent access DNS?

s131
00:09:28.457 --> 00:09:30.497
That's one of the homework questions.

s132
00:09:30.497 --> 00:09:31.337
All right.

s133
00:09:31.337 --> 00:09:33.417
Let's talk about data protection.

s134
00:09:33.417 --> 00:09:42.337
We want output filtering, content minimization, tenant isolation, query gateway, and memory integrity, the most exciting one.

s135
00:09:42.337 --> 00:09:54.657
So, in this case, there was a MCP server, and essentially it was pretty functional, but then it allowed you allowed the user to access any other user's data.

s136
00:09:54.657 --> 00:10:09.617
So, this actually very common incident that you see like cross agent sorry, cross data of various customers access, and this was running for a month unnoticed.

s137
00:10:09.617 --> 00:10:10.738
So, what do we want?

s138
00:10:10.738 --> 00:10:15.058
We want hard tenancy isolation, right?

s139
00:10:15.058 --> 00:10:20.618
As hard as possible, think like, you know, AWS accounts or, you know, let's say GCP projects.

s140
00:10:20.618 --> 00:10:24.578
You want workload identity tightly scoped to that domain.

s141
00:10:24.578 --> 00:10:29.258
You want data provenance tagging and you want audit trail.

s142
00:10:29.338 --> 00:10:43.978
Okay, so this one is pretty novel attack vector and this is a hidden instruction in a document which instructed to essentially remember that when the agent sees a secret,

s143
00:10:43.978 --> 00:10:47.418
make sure to send it to this DNS domain.

s144
00:10:47.418 --> 00:10:55.662
So, it's pairing the DNS exfiltration with essentially sleeper agent instructions, right?

s145
00:10:55.662 --> 00:10:56.538
[snorts]

s146
00:10:56.538 --> 00:11:03.978
So, now we need to think about how do we check memory for any of those malicious instructions?

s147
00:11:03.978 --> 00:11:11.818
I hear a security product from a vendor coming up very soon right after this talk to do the memory integrity checks.

s148
00:11:11.818 --> 00:11:13.658
Um I'm just kidding.

s149
00:11:13.658 --> 00:11:18.218
I'm definitely giving instructions to the agents.

s150
00:11:18.218 --> 00:11:20.818
Uh ephemeral runtime, right?

s151
00:11:20.818 --> 00:11:24.058
Ideally, you want to start fresh, no memories whatsoever.

s152
00:11:24.058 --> 00:11:28.898
Uh we want injection protection, isolation, and circuit breakers.

s153
00:11:28.898 --> 00:11:32.138
And finally, let's talk about the governance of observability.

s154
00:11:32.138 --> 00:11:34.538
We definitely want an audit trail.

s155
00:11:34.538 --> 00:11:36.338
Guardrails outside.

s156
00:11:36.338 --> 00:11:40.378
Circuit breakers, isolation verification of a serial tasting.

s157
00:11:40.378 --> 00:11:53.178
So, um in this case, this really sad incident, somebody instructed the agent, we are in code freeze, don't touch prod.

s158
00:11:53.178 --> 00:11:55.898
And look at this, understood, thumbs up.

s159
00:11:55.898 --> 00:11:59.458
That's very trustworthy, right?

s160
00:11:59.938 --> 00:12:01.458
And what happened?

s161
00:12:01.458 --> 00:12:04.578
Agent run drop database production.

s162
00:12:04.578 --> 00:12:09.138
All production also was dev, too, by the way.

s163
00:12:09.578 --> 00:12:11.818
Very sad.

s164
00:12:11.818 --> 00:12:17.579
A lots of uh uh people uh I actually had a friend by the way call me and say like what do I do?

s165
00:12:17.579 --> 00:12:19.939
How do I recover my database?

s166
00:12:19.939 --> 00:12:26.739
So we want circuit breakers on anything that is catastrophic, right?

s167
00:12:26.739 --> 00:12:30.099
We want human approval for any high value actions, right?

s168
00:12:30.099 --> 00:12:34.099
We want scoped credentials to only the actions we want the agent to do.

s169
00:12:34.099 --> 00:12:36.619
We definitely want tenant isolation.

s170
00:12:36.619 --> 00:12:39.899
Imagine it dropped all the tables.

s171
00:12:39.899 --> 00:12:41.339
All right.

s172
00:12:41.339 --> 00:12:45.219
So in this case this sounds very elementary.

s173
00:12:45.219 --> 00:12:47.819
Okay, so you had like a fake MCP on NPM.

s174
00:12:47.819 --> 00:12:50.499
What else is new here, right?

s175
00:12:50.499 --> 00:12:52.979
But it was 15 clean releases.

s176
00:12:52.979 --> 00:12:54.739
It's very useful utility.

s177
00:12:54.739 --> 00:13:04.539
And the attacker on the version 16 added a line on BCC is saying basically like send me by the way all the emails on BCC line to me.

s178
00:13:04.539 --> 00:13:06.539
Okay?

s179
00:13:06.739 --> 00:13:13.459
So this is kind of a classic supply chain verification, but I would say like don't just do supply chain verification.

s180
00:13:13.459 --> 00:13:15.499
Do delay ingest.

s181
00:13:15.499 --> 00:13:18.939
Don't ingest every package on day zero.

s182
00:13:18.939 --> 00:13:22.659
Ingest it maybe in a week or two.

s183
00:13:22.659 --> 00:13:29.739
Um you want a deterministic invocation broker least privilege tools, egress controls.

s184
00:13:29.739 --> 00:13:31.699
All right, so let's recap.

s185
00:13:31.699 --> 00:13:34.899
We would be talking about defense in depth for generative AI.

s186
00:13:34.899 --> 00:13:36.779
This is your toolbox.

s187
00:13:36.779 --> 00:13:46.459
You want to make sure that you have identity and trust, prompting input defense, tool and action governance, execution, isolation, data protection, governance, observability.

s188
00:13:46.979 --> 00:13:49.979
And as I said, this is kind of like eating your vegetables stalks.

s189
00:13:49.979 --> 00:13:52.019
So here's your vegetables.

s190
00:13:52.019 --> 00:13:57.059
Identity needs to be short-lived tokens, one per agent.

s191
00:13:57.059 --> 00:14:01.819
Input, outside text cannot give commands.

s192
00:14:01.819 --> 00:14:05.739
Tools syntax is not a control.

s193
00:14:05.739 --> 00:14:11.499
Isolation, default deny means everything including DNS.

s194
00:14:11.499 --> 00:14:15.261
Data, check the tenant on every query.

s195
00:14:15.261 --> 00:14:20.581
Governance prompts us deterministic code enforces.

s196
00:14:21.061 --> 00:14:25.461
Three priorities starting today.

s197
00:14:25.461 --> 00:14:27.021
Lock down your agents.

s198
00:14:27.021 --> 00:14:29.661
Use the rule of two.

s199
00:14:29.661 --> 00:14:39.501
Every tool call needs to go through some sort of deterministic broker and human sign off for any high value actions.

s200
00:14:39.501 --> 00:14:43.981
Second, one agent, one identity.

s201
00:14:44.021 --> 00:14:46.781
Kill any long lift keys.

s202
00:14:46.781 --> 00:14:52.221
Everything needs to be tight scoped tokens, short lift audience bound tokens.

s203
00:14:52.221 --> 00:14:56.021
Box the runtime but pair it up with default deny egress.

s204
00:14:56.021 --> 00:14:58.621
That means DNS as well.

s205
00:14:58.621 --> 00:15:03.261
Make sure there are budgets that halt, not just alert.

s206
00:15:04.621 --> 00:15:06.941
Finally, I will leave you with this.

s207
00:15:06.941 --> 00:15:10.381
I Who does the spec driven development here?

s208
00:15:10.381 --> 00:15:12.181
Well, yeah.

s209
00:15:12.181 --> 00:15:12.541
Got one.

s210
00:15:12.541 --> 00:15:13.341
Okay.

s211
00:15:13.341 --> 00:15:16.981
Make sure your spec includes security requirements.

s212
00:15:16.981 --> 00:15:18.461
And this is just a sample.

s213
00:15:18.461 --> 00:15:31.221
You can ping me on LinkedIn um for more comprehensive one and I also have no book alarm that explains a little more concepts if you were confused about some of them.

s214
00:15:31.221 --> 00:15:34.621
So, please connect on LinkedIn.

s215
00:15:34.621 --> 00:15:36.421
Build fast, deploy securely.

s216
00:15:36.421 --> 00:15:37.970
Thank you.

s217
00:15:37.970 --> 00:15:39.970
[applause]