WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=XjI-AR4pt7Y

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/XjI-AR4pt7Y.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:13.360 --> 00:00:14.120
Hi everyone.

s3
00:00:14.120 --> 00:00:22.880
So, my name is Lavina D'Mello and I'm a senior software developer at Nvidia and I work on the deep learning infrastructure team.

s4
00:00:22.880 --> 00:00:26.960
So, the title here is a bit of provocation.

s5
00:00:26.960 --> 00:00:39.400
What I really mean and what I want you to leave with is uh for the next 15 minutes, I'm going to make one uncomfortable argument, which is almost everything

s6
00:00:39.400 --> 00:00:45.360
that is breaking in the production ML security isn't some exotic AI attack.

s7
00:00:45.360 --> 00:00:52.120
It's the same boring infrastructure mistakes that we supposedly fixed years ago.

s8
00:00:52.120 --> 00:01:02.840
So, now once uh you look across the research from the whole field, the pattern is very easy, like it's very difficult to unsee.

s9
00:01:02.840 --> 00:01:04.800
Let me start with a story.

s10
00:01:04.800 --> 00:01:12.320
So, into in 2023, security researchers went looking at Ray clusters.

s11
00:01:12.320 --> 00:01:17.560
And Ray is one of the most popular framework for distributed ML.

s12
00:01:17.560 --> 00:01:26.200
So, what they found out over there was there were thousands of clusters that were sitting open on the internet.

s13
00:01:26.200 --> 00:01:33.960
What that means is the dashboards were open and there were job APIs that were open.

s14
00:01:33.960 --> 00:01:35.920
And why that happened?

s15
00:01:35.920 --> 00:01:48.280
Because authentication was off by default and somebody just forgot to turn it on while putting them ML in them uh in production.

s16
00:01:53.600 --> 00:01:55.936
So, I'm a little bit short, you know.

s17
00:01:55.936 --> 00:01:57.160
[laughter]

s18
00:01:57.160 --> 00:01:59.040
So, I need this.

s19
00:01:59.040 --> 00:02:00.680
Thank you.

s20
00:02:00.680 --> 00:02:01.560
Okay.

s21
00:02:01.560 --> 00:02:05.520
So, uh the exposure at that time was a lot.

s22
00:02:05.520 --> 00:02:08.000
Like, it was over billion dollars.

s23
00:02:08.000 --> 00:02:10.360
So, now let's sit with this part.

s24
00:02:10.360 --> 00:02:18.000
This was not a zero-day attack, and it was not a clever new attack uh on a neural network.

s25
00:02:18.000 --> 00:02:26.720
So, what happened was somebody just forgot to turn the default setting on while putting them into the production environment.

s26
00:02:26.720 --> 00:02:34.160
So, it just depends upon what the configurations are on different deployment environments in the infrastructure.

s27
00:02:34.160 --> 00:02:44.840
And when we look at the breaches that actually make the news, what we see, this is not a rule, it's not an exception, but it's almost never the fancy adversarial

s28
00:02:44.840 --> 00:02:45.800
attack.

s29
00:02:45.800 --> 00:02:51.080
What is happening is it's maybe like somebody exposed an API key.

s30
00:02:51.080 --> 00:02:55.160
What happened is maybe some account was just over-privileged.

s31
00:02:55.160 --> 00:03:00.600
Maybe some more model weight was uh in a public bucket.

s32
00:03:00.600 --> 00:03:07.080
And the whole breach, it's it was just like somebody forgot to flip.

s33
00:03:07.080 --> 00:03:08.760
So, how did we get here?

s34
00:03:08.760 --> 00:03:10.560
Let's rewind a little bit.

s35
00:03:10.560 --> 00:03:16.440
So, what used to happen is a classic 2008 application was deterministic.

s36
00:03:16.440 --> 00:03:20.519
So, we could reason what the code would exactly do.

s37
00:03:20.519 --> 00:03:25.640
You know, it had a fixed parameter, and it served one tenant.

s38
00:03:25.640 --> 00:03:30.880
Security researchers or security team had a clear mental model.

s39
00:03:30.880 --> 00:03:34.760
They knew like what they need to secure.

s40
00:03:34.760 --> 00:03:38.640
And like great tools were built around that time.

s41
00:03:38.640 --> 00:03:43.720
So, later what happened is we put this machine learning models into production.

s42
00:03:43.720 --> 00:03:47.360
And we quietly broke all of the three assumptions.

s43
00:03:47.360 --> 00:03:50.959
Because behaviors became probabilistic.

s44
00:03:50.959 --> 00:03:52.840
Correct is a distribution now.

s45
00:03:52.840 --> 00:04:03.200
It is not one branch like we say yes or no. And then we are protecting the weights which the weights itself are copyable.

s46
00:04:03.200 --> 00:04:08.160
And the leaks are straight through the API that we are trying to serve.

s47
00:04:08.160 --> 00:04:18.560
And to make it more affordable, we pack everything onto a shared multi-tenant GPU where other security risk come into picture.

s48
00:04:18.560 --> 00:04:22.760
And now the LLM stack itself blows up wide open.

s49
00:04:22.760 --> 00:04:28.840
And prompts become control flow and answers come from retrieval corpora.

s50
00:04:28.840 --> 00:04:32.440
And agents are going to take the actions for us.

s51
00:04:32.440 --> 00:04:34.240
So here is the thing.

s52
00:04:34.240 --> 00:04:42.560
We changed everything about the stack when we moved to machine learning, but we forgot to change the security assumptions.

s53
00:04:42.560 --> 00:04:47.600
So we are running a 2028 playbook on a 2026 system.

s54
00:04:47.600 --> 00:04:50.480
That's what the whole talk is about.

s55
00:04:50.480 --> 00:04:57.800
So few things, five things stand out across all of these and I have tried to boil them down into three core ideas.

s56
00:04:57.800 --> 00:05:00.000
Uh and here they are.

s57
00:05:00.000 --> 00:05:02.560
So first one is misconfigurations.

s58
00:05:02.560 --> 00:05:05.520
So lot of the times misconfigurations happen.

s59
00:05:05.520 --> 00:05:10.080
It is not the missing features that are number one failure reason.

s60
00:05:10.080 --> 00:05:17.160
Second is the defenses that we are actually reading, they might not actually survive into production.

s61
00:05:17.160 --> 00:05:25.120
And third one is we have to secure ML like infrastructure and it's not like securing just a model, you know?

s62
00:05:25.120 --> 00:05:32.520
And if we if you forget everything about this talk, I want you to remember these three points over here.

s63
00:05:32.520 --> 00:05:35.800
Uh and I'll share a few couple of things at the end.

s64
00:05:35.800 --> 00:05:38.320
Uh Okay.

s65
00:05:38.320 --> 00:05:41.960
So over here is a map I want you to carry with you.

s66
00:05:41.960 --> 00:05:46.120
It's four pillars about the defenses in depth.

s67
00:05:46.120 --> 00:05:52.000
So at the bottom we can see the at the bottom in the black we can see the infrastructure security.

s68
00:05:52.000 --> 00:05:59.200
Over here comes into picture the containers, the networks, the GPU virtualization and encryption.

s69
00:05:59.200 --> 00:06:04.320
On top So, this infrastructure security is the foundation.

s70
00:06:04.320 --> 00:06:19.520
And on top of it sits the access control like who is allowed to do what and then is the runtime security which is checking uh and watching the traffic while the system is life.

s71
00:06:19.520 --> 00:06:33.480
So, they feed each other and wrapping all of it uh to day-to-day practice like um how the team works plus compliance, incident response, and honestly the culture.

s72
00:06:33.480 --> 00:06:42.680
So, one thing to remember over here is if uh infrastructure is the foundation and failures down there can cascade up.

s73
00:06:42.680 --> 00:06:51.400
So, if we have a perfect access control policy, it's just a decoration if the cluster underneath is wide open.

s74
00:06:51.400 --> 00:06:53.680
So, yeah.

s75
00:06:53.680 --> 00:06:55.640
So, those are the four levels.

s76
00:06:55.640 --> 00:07:00.000
And this is a quick map of what we are defending against.

s77
00:07:00.000 --> 00:07:02.200
So, there are six categories.

s78
00:07:02.200 --> 00:07:05.040
The two are about the model itself.

s79
00:07:05.040 --> 00:07:10.360
So, where we have the adversarial inputs and we have the model extraction.

s80
00:07:10.360 --> 00:07:23.480
Over here the primary defenses should be uh about how we do the input validation, how we do the adversarial detection, how we do the rate limiting, API auth, and query analysis.

s81
00:07:23.480 --> 00:07:26.480
The next two are about the data and integrity.

s82
00:07:26.480 --> 00:07:31.440
Over there comes the poisoning and supply chain and privacy breaches.

s83
00:07:31.440 --> 00:07:43.600
So, we should make sure that over there we make sure to secure the CI/CD, model signing, and provenance, access control, encryption, all of those comes into that picture.

s84
00:07:43.600 --> 00:07:57.160
And the last one, our classic infrastructure, where we take care of um basically container isolation, network segmentation, uh R back, and behavior analysis.

s85
00:07:57.160 --> 00:08:09.200
So, if we notice one thing over here, uh the green ones, infrastructure compromise and insiders, those are exactly where the real breaches keep landing.

s86
00:08:09.200 --> 00:08:18.640
And they are the cheapest to get wrong by accident, which brings me to the core idea number one, which is mis- misconfiguration, not missing feature.

s87
00:08:18.640 --> 00:08:22.240
So, those that is one of the number one failure mode.

s88
00:08:22.240 --> 00:08:24.040
Um okay.

s89
00:08:24.040 --> 00:08:28.000
So, let me walk you through one of the research.

s90
00:08:28.000 --> 00:08:35.080
What happens is researchers audited 50 real production setups running machine learning.

s91
00:08:35.080 --> 00:08:43.400
In 78% of them, what the researchers found out was at least one critical security mistake.

s92
00:08:43.400 --> 00:08:48.360
And and And here's why that mattered beyond engineering.

s93
00:08:48.360 --> 00:09:03.920
So, that was a misconfiguration, and the difference between a normal Tuesday and something having a headline about our company in the news is this kind of incident, where there could be like a critical misconfiguration

s94
00:09:03.920 --> 00:09:05.240
mistake.

s95
00:09:05.240 --> 00:09:11.280
And And the same three problems kept showing up in this research study.

s96
00:09:11.280 --> 00:09:15.520
First was access controls were left wide open.

s97
00:09:15.520 --> 00:09:19.240
So, what happens when an access controls are left wide open?

s98
00:09:19.240 --> 00:09:23.720
Almost any account can do almost anything.

s99
00:09:23.720 --> 00:09:25.520
So, that is one of the problem.

s100
00:09:25.520 --> 00:09:29.080
Second is nothing sep- nothing was separated.

s101
00:09:29.080 --> 00:09:39.640
For example, different parts of the system So, once an attacker would get access to one part of the system, he could do anything in the whole system.

s102
00:09:39.640 --> 00:09:43.880
So, there is the thing like separating different parts of the system.

s103
00:09:43.880 --> 00:09:49.000
And the third one were the passwords and trained models.

s104
00:09:49.000 --> 00:09:54.040
So, themselves they were sitting into a storage that anybody could reach.

s105
00:09:54.040 --> 00:09:55.680
So, why does this keep happening?

s106
00:09:55.680 --> 00:10:05.240
So, because this production protections are built by security experts for security experts and then they get handed to the ML teams.

s107
00:10:05.240 --> 00:10:14.680
And the responsibility of ML team is um basically making the model accurate, not configuring the infrastructure.

s108
00:10:14.680 --> 00:10:17.480
So, the protection is right there in the platform.

s109
00:10:17.480 --> 00:10:23.600
It just sometimes get switched off by some configurations.

s110
00:10:25.960 --> 00:10:26.440
Okay.

s111
00:10:26.440 --> 00:10:33.720
So, now moving to the core idea two, which is defenses you read about don't survive into the production.

s112
00:10:33.720 --> 00:10:37.400
So, here is here is the thing.

s113
00:10:37.400 --> 00:10:40.640
Every control costs us something.

s114
00:10:40.640 --> 00:10:45.360
And in ML that something is latency and throughput.

s115
00:10:45.360 --> 00:10:49.320
The two things that SLA is made up of.

s116
00:10:49.320 --> 00:10:51.000
Let's watch the colors.

s117
00:10:51.000 --> 00:11:03.320
So, the green tier over here, the basics like logins, input checking, costs do those cost under about 8% and that should always be there.

s118
00:11:03.320 --> 00:11:04.920
No debate over there.

s119
00:11:04.920 --> 00:11:07.160
The second one is the orange tier.

s120
00:11:07.160 --> 00:11:16.280
So, this is like the heavier isolation that walls around that that walls workloads of each other.

s121
00:11:16.280 --> 00:11:19.600
So, here it runs 10 to 20%.

s122
00:11:19.600 --> 00:11:28.200
So, we can use it selectively like for sensitive and untrusted workloads, what we can do is we can decide if we want it or not.

s123
00:11:28.200 --> 00:11:31.960
And the purple ones is catching malicious input in real time.

s124
00:11:31.960 --> 00:11:34.680
So, this is the most expensive one.

s125
00:11:34.680 --> 00:11:42.600
Over here, it can cost like I don't know, 15 to 30% and it's one of the thing that researchers love the most.

s126
00:11:42.600 --> 00:11:49.480
But, because we cannot slow every request down, so it's a non-starter.

s127
00:11:49.560 --> 00:11:56.800
But, what really matters is how much it cost depends almost entirely on how well we build it.

s128
00:11:56.800 --> 00:12:04.120
So, the same security check check, it can be done sloppily and it can double our response time.

s129
00:12:04.120 --> 00:12:11.560
But, if it is done well, like considering all of these things, then it's fair, like the balance is fair.

s130
00:12:11.560 --> 00:12:20.760
So, the guidance team actually need isn't should I do this, it's like how do I do it without slowing everything down.

s131
00:12:20.760 --> 00:12:30.920
And then we can run the cheap controls everywhere and we can save the expensive ones for the heavier risk systems.

s132
00:12:31.960 --> 00:12:34.600
So, here is the the next slide.

s133
00:12:34.600 --> 00:12:36.920
The ecosystem isn't production ready yet.

s134
00:12:36.920 --> 00:12:43.400
So, what I meant is um So, there is the first gap that we will see.

s135
00:12:43.400 --> 00:12:49.000
The security tooling for ML is years behind the rest of the software.

s136
00:12:49.000 --> 00:13:04.080
So, regular software teams, they solved a lot of this decades ago with automated security scanning, proper passport password management, and then we haven't carried it all over to ML yet.

s137
00:13:04.080 --> 00:13:10.080
Even today, there's a good share of early warning systems for ML specific threats.

s138
00:13:10.080 --> 00:13:13.760
So, the second gap is in the expertise silos.

s139
00:13:13.760 --> 00:13:22.800
So, security team don't speak ML, ML teams don't speak security, and the operation teams don't know how the model behaves.

s140
00:13:22.800 --> 00:13:26.880
So, there is a gap over here and the problem leaves in the gaps.

s141
00:13:26.880 --> 00:13:30.480
So, between those gaps where nobody has a full picture.

s142
00:13:30.480 --> 00:13:33.640
So, there's a little bit of mismatch over here.

s143
00:13:33.640 --> 00:13:37.760
And underneath both is a gap between the research and reality.

s144
00:13:37.760 --> 00:13:44.120
So, research is busy defending against tiny invisible tweets.

s145
00:13:44.120 --> 00:14:01.800
But to an image that while the real world, we are fighting against the stolen password and then people are misusing access that they already have and it studies one model on its own while we are running dozens of them

s146
00:14:01.800 --> 00:14:03.760
on shared system.

s147
00:14:03.760 --> 00:14:09.640
So, what it lacks are one team can actually deploy everything.

s148
00:14:09.640 --> 00:14:11.800
So, there is this core idea number three.

s149
00:14:11.800 --> 00:14:13.080
It's the constructive one.

s150
00:14:13.080 --> 00:14:18.840
So, if the problem is infrastructure, we have to secure it like infrastructure, not like a model.

s151
00:14:18.840 --> 00:14:30.840
And two practical tools for that are a maturity model to know where we stand and a short list of how to fix it first.

s152
00:14:31.160 --> 00:14:34.200
So, here is the next slide.

s153
00:14:34.200 --> 00:14:38.640
What we have to do is how to figure out where we actually stand.

s154
00:14:38.640 --> 00:14:45.160
It's a maturity model that will help us map onto the NIST AI risk management framework.

s155
00:14:45.160 --> 00:14:51.640
So, there will be four levels, each tied to an overhead budget and the context.

s156
00:14:51.640 --> 00:14:57.960
So, on the level one, we can see that under 5% overhead is the bare basics.

s157
00:14:57.960 --> 00:15:00.920
Fine for test environment, not for production.

s158
00:15:00.920 --> 00:15:10.200
So, the level two 5 to 10% is a real baseline like proper logins, encryptions, separating our networks, basic monitoring.

s159
00:15:10.200 --> 00:15:12.760
This is the minimum bar for production.

s160
00:15:12.760 --> 00:15:21.400
And the level three adds more advanced controls that regulated industries like health care and finance needs.

s161
00:15:21.400 --> 00:15:31.760
So, on this part, most of teams teams believe that they are at level three, but uh they are actually either at level one or two, like it really depends

s162
00:15:31.760 --> 00:15:35.080
how we have set up everything.

s163
00:15:38.800 --> 00:15:44.480
So, on this slide, I want to discuss about the three misconfiguration behind most of the risk.

s164
00:15:44.480 --> 00:15:50.320
So, the default setup often gives every account a full admin rights.

s165
00:15:50.320 --> 00:15:53.880
So, that is what have been observed in most of the research studies.

s166
00:15:53.880 --> 00:16:00.320
With credentials that never expire, so there are there have been credentials that do not have any expiry.

s167
00:16:00.320 --> 00:16:04.240
And that can leak the key on its own.

s168
00:16:04.240 --> 00:16:11.280
And like each account like should do only what it needs to do.

s169
00:16:11.280 --> 00:16:16.120
And then the credentials to should expire very quickly.

s170
00:16:17.560 --> 00:16:21.760
Now, nothing stops one part of the system from reaching another part of the system.

s171
00:16:21.760 --> 00:16:27.680
So, if that can be tackled in a way uh then that can be solved.

s172
00:16:27.680 --> 00:16:32.800
Like attacker gets in one part of the system and then they get it can do everything.

s173
00:16:32.800 --> 00:16:37.480
So, that's uh like a flat network kind of issue.

s174
00:16:37.480 --> 00:16:38.400
Then the fix.

s175
00:16:38.400 --> 00:16:44.360
So, what we can do to fix this is we can wall the parts from each other.

s176
00:16:44.360 --> 00:16:50.160
And we can require a verified identity between services, not just network access.

s177
00:16:50.160 --> 00:16:55.360
So, there is an another issue called the exposed secrets and models.

s178
00:16:55.360 --> 00:17:02.280
So, over there passwords could be hardcoded in the code, model files sitting in the open storage.

s179
00:17:02.280 --> 00:17:10.680
So, the fix could be like a proper secret manager, encryption, and automatic scanning before before all of this ships.

s180
00:17:10.680 --> 00:17:13.640
So, this is nothing but like a basic hygiene.

s181
00:17:13.640 --> 00:17:20.439
And this kind of hygiene can prevent breaches that actually make the news.

s182
00:17:21.760 --> 00:17:24.240
So, uh here's the practical stuff.

s183
00:17:24.240 --> 00:17:31.160
Uh so, now a couple of personal takes on where this is heading because the ground is moving very faster under us.

s184
00:17:31.160 --> 00:17:34.720
So, this is the part uh that's least settled.

s185
00:17:34.720 --> 00:17:37.080
And four things are on my mind right now.

s186
00:17:37.080 --> 00:17:39.400
So, one is prompt injection.

s187
00:17:39.400 --> 00:17:49.120
So, the model can't reliably tell the difference between our instructions and someone else's input.

s188
00:17:49.120 --> 00:17:54.800
So, a cleverly worded message can quickly hijack the whole system.

s189
00:17:54.800 --> 00:18:09.160
So, rag poisoning, if our model pulls answers from a document store, and if the document store is poisoned by somebody or somehow, then what happens is it can poison like everything.

s190
00:18:09.160 --> 00:18:11.840
And nobody has to retain anything, right?

s191
00:18:11.840 --> 00:18:15.200
So, GPUs So, next is the GPU side channel.

s192
00:18:15.200 --> 00:18:20.600
So, where we can pack multiple customers on one GPU in order to save money.

s193
00:18:20.600 --> 00:18:25.400
That time information can leak from one customer to the other.

s194
00:18:25.400 --> 00:18:27.240
And another is the supply chain.

s195
00:18:27.240 --> 00:18:37.920
So, what happens over there is we download models and add-ons from public sources without uh really checking where they come from.

s196
00:18:37.920 --> 00:18:39.800
So, that's another issue.

s197
00:18:39.800 --> 00:18:45.520
And the honest caveat over here is uh all of this moves very fast.

s198
00:18:45.520 --> 00:18:48.280
Every week something or the else comes up.

s199
00:18:48.280 --> 00:18:58.160
So, we treat it as a direction, not a gospel, and the defenses are immature right now because uh the threats keep evolving.

s200
00:18:59.560 --> 00:19:00.000
Okay.

s201
00:19:00.000 --> 00:19:03.000
So, three uh So, let's bring it together.

s202
00:19:03.000 --> 00:19:07.520
If you forget everything I said today, here are the three takeaways.

s203
00:19:07.520 --> 00:19:13.240
So, one is securing ML is an infrastructural problem, not a model problem.

s204
00:19:13.240 --> 00:19:19.280
So, we have to spend our budget where the breaches actually are happening.

s205
00:19:19.280 --> 00:19:22.160
Second, the field has enough defenses.

s206
00:19:22.160 --> 00:19:25.160
It just need deployable ones.

s207
00:19:25.160 --> 00:19:28.920
The frontier isn't a new attack defense pair.

s208
00:19:28.920 --> 00:19:34.280
It's making the existing controls run with proper production overhead.

s209
00:19:34.280 --> 00:19:38.880
And that's the engineering problem, which is a good news for a room full of engineers.

s210
00:19:38.880 --> 00:19:47.920
And the third one is we have to fix the misconfigurations, like over-privileged accesses, flat networks, and exposed secrets.

s211
00:19:47.920 --> 00:19:52.520
So, then climb the maturity model on purpose.

s212
00:19:52.520 --> 00:20:00.760
And to close the loop on the title, your LLM stack really is a 2008 database with better marketing.

s213
00:20:00.760 --> 00:20:09.360
So, secure it like a database, lock down accesses, segment the network, and protect the data at rest.

s214
00:20:09.360 --> 00:20:11.040
So, we already know how to do that.

s215
00:20:11.040 --> 00:20:14.480
We just have to do it actually in a better way.

s216
00:20:14.480 --> 00:20:15.240
That's all.

s217
00:20:15.240 --> 00:20:17.840
So, thank you.

s218
00:20:18.314 --> 00:20:20.314
[applause]