Agentic Security: Permissions, Provenance, and the Agent Supply Chain — Steve Yegge, Gas Town

AI Engineer · 22 min · 361 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

1. 00:01[music]
2. 00:12All right.
3. 00:13Hey everybody.
4. 00:14Uh, yeah, yeah, yeah, yeah.
5. 00:17Okay.
6. 00:18I have about 18 minutes.
7. 00:19Um, I have a slide deck here that that Claude put together for me, but Claude still can't do very good slide decks, so I apologize for that.
8. 00:28Just just just focus on me, right?
9. 00:30Right here.
10. 00:33This is like the most fun thing to do every now.
11. 00:35If I if I start like running out of time, can can you like call time for me?
12. 00:41All right, perfect.
13. 00:42So, I'm Steve Yege.
14. 00:45I am here on behalf of Sneak today.
15. 00:48I am not getting paid for this talk or anything like that.
16. 00:51They were going to nice enough to buy me a ticket to the conference, which is cool, but I'm mostly here because I wanted to hang out with all of you, and I don't bite, and feel free to come and say hi afterwards.
17. 01:01I I wandered the halls amazed at all the brand logos.
18. 01:04Um, I'm up here because like the title of my talk is like "Agentic Security", but the real title of my talk is "Be Scared".
19. 01:18See, I Come on.
20. 01:21Come on.
21. 01:21Level with me.
22. 01:22Who here is scared?
23. 01:24That's good, but you're all in the security talks.
24. 01:27Of course you are.
25. 01:27The problem is they aren't out there.
26. 01:31Right?
27. 01:32I I went to Commonwealth I went to so many places last year, but I one of these big banks I was at, I was talking to their chief security architect in December,
28. 01:41and I was doing a Q&A, and talking about vibe coding, and everyone was like asking me questions, and I was you know, knocking them out of the park, and
29. 01:49he stands up real quiet at the end, and he goes, "Yeah, so he goes, "If everyone's shipping code at the same at sorry, at 10 times faster and the defect rate stays the same, the security defect, right?
30. 02:04The the vulnerability rate.
31. 02:10Then that doesn't that mean that the defects surface goes up by 10x?
32. 02:16And I it hit me so hard.
33. 02:18I I sank down to my knees.
34. 02:19I was just like what are we going to do about this?
35. 02:23Because it's a really really important point.
36. 02:26Cuz the subtle implied question is not if the defect rate stays the same, the defect rate's going to get worse.
37. 02:34A lot worse with AIs writing the code.
38. 02:39And so I didn't have an answer for him.
39. 02:43I actually have a partial answer for you today and I'll share it with you, but it's only a partial answer.
40. 02:49The real answer is you have to be scared of what's coming.
41. 02:53All right?
42. 02:55And it's becau it's bec it's it's it's Do I have a slide about this?
43. 02:59Yeah.
44. 03:00So it's not just that you're just like putting in more of the same defects cross-site scripting and blah blah blah.
45. 03:08You're still putting those in.
46. 03:09Fable wrote a an XSS vulnerability during the short time that I had with it.
47. 03:15Not its fault.
48. 03:16We'll talk about it in a minute.
49. 03:19[snorts]
50. 03:19But those are the old ones.
51. 03:22All right, we are we all know how to fix those.
52. 03:24There's new vulnerability types and new attack surfaces coming and they're here and many of them are incredibly well polished.
53. 03:31Like what's an example?
54. 03:33You guys know about slop squatting?
55. 03:36Right?
56. 03:36Where the AI hallucinates a package name.
57. 03:39Let's say that you want a graph database and so you're like, "Okay, I'm going to use this graph database."
58. 03:43And the AI goes, "Oh yeah, I know.
59. 03:45It's you know, it's graphy 123."
60. 03:47And it goes off to the package manager and downloads graphy 123.
61. 03:50And it builds and it runs, and the tests pass, and it looks right, but what it downloaded was a backdoor.
62. 04:00Because that graphy123 wasn't a real package, but somebody noticed that the LLMs are hallucinating its name, and they uploaded one that does exactly the same thing as the one it thought it was getting,
63. 04:10plus a vulnerability.
64. 04:11Bonus.
65. 04:13Yeah?
66. 04:14Is that scary?
67. 04:16Yeah, it should be.
68. 04:18It should be.
69. 04:19How do you even detect that that's happening?
70. 04:22So, so we're entering a world where everything you write every bit of code that you generate is going to have to get far more security scrutiny than it's ever had before.
71. 04:41Okay?
72. 04:41It's just you're not It's hard for me to convey how scared I am.
73. 04:47All right?
74. 04:49But let's start with where it gets generated.
75. 04:51Now, when I worked at Google, I worked really close with the TAP team.
76. 04:54They did tests.
77. 04:55The test automation platform, yeah?
78. 04:58And they uh So, they ran all of the unit tests and integration tests at Google.
79. 05:02They had a massive fleet, you know, and and they learned stuff about how bugs work.
80. 05:07And how bugs work is they have a lot they have like a life cycle where if you see the bug right away, you'll fix it.
81. 05:15And the longer the time goes for when you see a warning or some sort of issue, the longer that passes Okay?
82. 05:22It's got this sort of half-life of urgency, and all of a sudden, eh, it ain't really biting anyone anymore, okay?
83. 05:28This is how we handle all of our bugs.
84. 05:30At at Google, they recognized that this is such a human nature phenomenon that they worked really hard to to move the reporting of bugs as you were typing.
85. 05:43Because that's when you're most likely to fix it.
86. 05:46If you make a bug, and it goes, "By the way, there's a divide by zero here or there's a backdoor or a vulnerability or whatever."
87. 05:51You'll You'll fix it right there.
88. 05:53But, if it gets to code review time, you're like, "Is it really worth it?"
89. 05:56Right?
90. 05:57[snorts]
91. 05:58And the problem, folks, is that that works for all classes of bugs except for security.
92. 06:06There's no half-life on it biting you.
93. 06:08It's not like, "Oh, because users aren't getting bothered by this security vulnerability that it's not a problem over time."
94. 06:15The problem compounds over time.
95. 06:18Yeah?
96. 06:19So, you have to treat this class of vulnerabilities the way that Google treated their top vulnerabilities at the time, which is to surface them at the developer's fingertips.
97. 06:31What if the developer doesn't have any fingers?
98. 06:34I don't know how how many fingers the LLMs have.
99. 06:36[snorts]
100. 06:37Um then you need to surface it to the LLMs.
101. 06:44But, wait.
102. 06:45You say, "Wait.
103. 06:45Wait.
104. 06:46Wait.
105. 06:46Wait.
106. 06:46Wait.
107. 06:47Wait.
108. 06:47Fable's really smart."
109. 06:48Or at least it seemed that way for the 2 days I got to use it.
110. 06:52Uh can't Fable just write secure code?
111. 06:54Right?
112. 06:55I mean, come on.
113. 06:57Come on.
114. 06:57You You all know secure I mean, you're all here in this room, right?
115. 07:00You all know security is an arms race.
116. 07:03One that never ends.
117. 07:04One that's going up exponentially with Moore's law.
118. 07:06One that's going to get real uncomfortable when quantum comes along.
119. 07:09Thank goodness that's like 5 to 7 years away.
120. 07:13Uh my buddy says 15, so maybe somewhere in between.
121. 07:16But, in the meantime, right?
122. 07:18LLMs are a real problem.
123. 07:20Yeah?
124. 07:21So, how do you surface How do you surface the vulnerabilities?
125. 07:25Well, first I want to figure out how to do it myself.
126. 07:27I have a game I've been working on for 30 years.
127. 07:30I just had Fable do a security hardening pass during the time I had it.
128. 07:34Went through and it did all my cloud hardening, and it found a bunch of credentials, and and all a bunch of stuff.
129. 07:42And it
130. 07:42[snorts]
131. 07:42started to give me these vibes like, yep, yep, Harding pass looking pretty good.
132. 07:47So, I ran sneak, right?
133. 07:48And I I don't have the numbers here.
134. 07:50Yeah, I didn't I didn't include the numbers cuz I'm done.
135. 07:54But, um uh it found 241 vulnerabilities, right?
136. 07:58Just a ton that Fable hadn't even thought to look for, right?
137. 08:03Uh and it's because look, so I I I've told people about the rule of five.
138. 08:09When you do things with LLMs, often you have to get them to do up to four to five reviews of the work that they did before it's like actually ready to ship.
139. 08:17And it's because their cognitive process is very similar to ours and it goes through a draft and then a revision and then polish and editing until it's, you know, it's finally ready to go.
140. 08:29It's like painting a wall.
141. 08:31Some things you don't just do all at one pass.
142. 08:33You do them in multiple passes, right?
143. 08:36So, um security is one of the So, what I found I wrote a book on vibe coding last year.
144. 08:48I did did more vibe coding anything than anyone, you know, two two years ago.
145. 08:53And and what I found was um that they're really good at doing one thing at a time.
146. 09:02Even the really good models like Fable, right?
147. 09:04Just because of this multi-pass painting a wall phenomenon.
148. 09:09You got to give them one task at a time, which means you can't give them security at the same time as you give them correctness.
149. 09:16They'll do a half-ass job of both.
150. 09:19And you don't want a half-ass job of either of those, it turns out.
151. 09:23So, you do it in two passes, right?
152. 09:25Now,
153. 09:25[snorts]
154. 09:26it five It's been five months now, but five months ago I wrote an essay called Software Survival 3.0 where I talked about what software has to do to survive
155. 09:33when LLMs can synthesize it all.
156. 09:35I don't know if any of you all saw that, but the basic the basic gist of it is that LLMs can synthesize any software that they want, but they're very lazy
157. 09:45in a good way, right?
158. 09:46Lazy in like they don't want to spend tokens if they don't have to, because that's money and power and bad for the planet and bad for your wallet and so on, right?
159. 09:54And so they use tools to help them whenever it can save tokens, right?
160. 10:01So tying it all together, if the LLMs are doing the coding and they're happy to use tools to help them offload cognition, you see where this is going?
161. 10:09Give them sneak.
162. 10:10Give them chain guard.
163. 10:12And I still think there's a missing piece in this picture that I'll tell you about at the end.
164. 10:19Um chain I don't know if you all know about chain guard.
165. 10:22Chain guard uh Chain guard is a supply chain that you sign up for and they give you images that have been pre-vetted to not have vulnerabilities and they update them.
166. 10:33So it's your inputs, okay?
167. 10:36And then sneak handles everything else.
168. 10:39The code that you write, the code that the LLM writes, the dependencies that you're pulling in from slot squatting, right?
169. 10:47The innocent stuff.
170. 10:48It can find My understanding is that sneak can actually find vulnerabilities that are proprietary, that only they know about because they're ahead of the CVE registry.
171. 10:57I There's some truth to that.
172. 10:59When I ran them on my code base, it didn't find any vulnerabilities that weren't already public CVEs, but damn, it was easy to [clears throat] use, right?
173. 11:09I think that a tool like sneak is going to give your LLMs superpowers, okay?
174. 11:17Cuz what you do is you add it as a pass to the prompt that you give them for whatever they're doing.
175. 11:24And say one last thing to look at and have them run your security analysis, all of the tools, get the open source ones, get the sneak one, get the chain guard one, get the all of them and have them
176. 11:37check each other's work, too.
177. 11:38Right?
178. 11:39If you want to get really serious about this on lunch time, right?
179. 11:42But secure your supply chain because Five Eyes is warning us.
180. 11:46Yeah?
181. 11:47My poor my poor game.
182. 11:48By the way, please please I just told you that my game has 241 vulnerabilities.
183. 11:53Don't go hack my game.
184. 11:54Give me a couple days to fix the bugs.
185. 11:55Yeah?
186. 11:56We all good here?
187. 11:57Good.
188. 11:58All right, but if you do hack it, you're going to bother like five players, all right?
189. 12:02[snorts]
190. 12:02Okay.
191. 12:03They're very loyal.
192. 12:04Um look, Five Eyes, which is like a bunch of right governing it's it's big countries that are have have their eye on on the cybersecurity, you know, landscape.
193. 12:14They just announced that it is now months, not years, until it starts happening.
194. 12:18It.
195. 12:21You all know what it is, right?
196. 12:23It is when open source models catch up to Mythos.
197. 12:30Does anybody here believe open source models are going to catch up to Mythos?
198. 12:35Interesting that it's about 50/50.
199. 12:38Anyone got a time frame in mind?
200. 12:42Who said December?
201. 12:43That's pretty accurate.
202. 12:47Cheater.
203. 12:48Yeah, it's about 7 months.
204. 12:49So, actually it's shrinking, so it's probably about 6 months now.
205. 12:52Yeah.
206. 12:54And Mythos is real real good at hacking your systems.
207. 12:59And and just just remember you can't trust it to automatically write good code any more than you can trust it to write elegant code by default.
208. 13:10That's a separate concern.
209. 13:12It's a separate pass.
210. 13:13You can't expect it to write performant code by default.
211. 13:17That's another pass.
212. 13:19You see what I'm saying?
213. 13:19You can't expect it to necessarily write the code according to your company coding standards, okay?
214. 13:24These are all passes that go through your code and I just want you to remember that security should be your first one and your last one.
215. 13:31Okay?
216. 13:32Give it extra.
217. 13:34Okay, and the last thing I wanted to talk to you about first of all, go do all this, and second of all, the last thing is really true true truthfully,
218. 13:43okay?
219. 13:44Dial it in here, folks.
220. 13:46Go to your families offline, like in person, and get your your code words refreshed.
221. 13:55Cuz another kind of scam that's coming along is you get a call from a family member who's in distress, and they need money, and it's very convincing, and there's a video of them, and you're going to need a way to distinguish them from from AI.
222. 14:07Okay?
223. 14:09It's months away, and some of your families are going to be slow to catch on to this stuff, but bank accounts will be drained.
224. 14:15I heard that Congress was given secret demos of draining bank accounts.
225. 14:19I've been scared of this for close to 2 years.
226. 14:23I heard one talk from a security researcher almost 2 years ago at ETLS Las Vegas, and he stood up in front of the crowd, and he said, "You're all not scared enough of what's coming."
227. 14:34It'll affect you personally, not just your company.
228. 14:37Okay?
229. 14:38So, that's my message to you.
230. 14:39It's not a message of hope and positivity today,
231. 14:42[laughter]
232. 14:43but it is a it is a message that that's that should be clear crystal clear is that there are tools, open-source tools, free tools, commercial tools, okay?
233. 14:53Techniques, practices, okay?
234. 14:56That you can use right now to get started on fighting in this arms race and protecting yourself.
235. 15:01And that's all I've got today.
236. 15:02Thank you.
237. 15:04[applause]
238. 15:09[applause]
239. 15:13Questions?
240. 15:17We ready to go?
241. 15:30Sweet.
242. 15:31Um just super simple.
243. 15:33What has surprised you recently?
244. 15:35In the world of AI coding.
245. 15:37What has surprised me in the world of AI coding?
246. 15:42Well, I'm not really super representative.
247. 15:44I spend a lot of my time trying to predict the future by like hammering agents really, really hard.
248. 15:49Yeah.
249. 15:51Um so um Uh you know, one of the surprises and I shouldn't have been surprised, but one of the surprises is that AI is moving faster than the world is moving.
250. 16:02Uh tech is moving faster than society can move.
251. 16:08And the surprises show up when friends, smart friends, resist uh you know, the inevitability of AI and they call it psychosis or they right?
252. 16:20Or they poo-poo it and they say, "Well, it'll never be actually smart."
253. 16:23Or whatever.
254. 16:23They they can't see the curve.
255. 16:25Right?
256. 16:26And uh and that surprises me.
257. 16:28Uh maybe it shouldn't.
258. 16:30Um it's for it shows a sort of tunnel vision.
259. 16:33I think people have a tendency to look about 3 months back and about 3 months forward and be like, "Oh, it looks pretty flat."
260. 16:39Right?
261. 16:40But but and so that that surprises me that people aren't honestly that people aren't more scared and that and by the same by the flip side that people aren't more excited by it.
262. 16:49Right?
263. 16:50Cuz you know once you actually, you know, once once you get it, I mean you you don't even want to be here.
264. 16:56You How many of you are running Claude code right now?
265. 16:59Most of you, right?
266. 17:00It's really fun.
267. 17:01So, you know, I mean like that's a surprise, too, that the world is pushing back so hard on that, right?
268. 17:07We're in an awkward phase.
269. 17:09We'll get through it.
270. 17:11Any other questions?
271. 17:13Woah.
272. 17:13All right.
273. 17:14Well, you pick.
274. 17:19Feel free to bail also.
275. 17:21You don't have to stay.
276. 17:22I big fan.
277. 17:24Um, what's the like most impressive uh thing you've seen Gastown do?
278. 17:28And like how much human intervention was involved or steering?
279. 17:30Oh, Gastown?
280. 17:32Yeah.
281. 17:33Gastown.
282. 17:34Gastown was a lot of fun in January.
283. 17:36Um, yeah, uh Gastown is a Beads machine and I still use Beads.
284. 17:42And I'm working I want to I was talking to Angie, I want to donate Beads to the Agentyc Foundation, you know.
285. 17:48We're going to we're going to put multiple backends on it.
286. 17:51Beads is a task tracker, right?
287. 17:53Beads is how you do Boris Cherney loops.
288. 17:56You know how Boris is like you shouldn't be prompting your agent?
289. 17:59Has anybody here actually like successfully How often do you get Claude to actually run all night for you?
290. 18:06Like for real run all night.
291. 18:08A few of you, right?
292. 18:10Like this is the next frontier, I think, of actually getting agents to run like for a long long time unsupervised.
293. 18:16You can do it with Beads by queuing up enough enough work and having them claim and all that.
294. 18:21And there are some other some other systems that'll do that.
295. 18:24It was really fun when Gastown did this for me automatically once.
296. 18:27I filed a whole bunch of Beads and they disappeared and I was like, "Oh, no, another bug.
297. 18:30My Beads disappeared."
298. 18:31And what had actually happened was that
299. 18:32[snorts]
300. 18:32one of the agents just found them and just implemented everything, right?
301. 18:36So I was like, "Whoa, I really like swarms now."
302. 18:39Yeah.
303. 18:40Fun times.
304. 18:41Does anybody here regularly work with more than 10 coding agents at once?
305. 18:48You see like not many, right?
306. 18:49The world is still in the we're still kind of like prompting and using a few here and there, right?
307. 18:54It's going to accelerate really fast next year.
308. 18:58Other questions?
309. 19:00You.
310. 19:01You can just say it.
311. 19:01Yeah.
312. 19:02Um, your talk focused a lot on vulnerabilities from agents writing code.
313. 19:07I'm curious what are your thoughts about agents taking actions
314. 19:13Yeah, so that was the third dimension that I really wanted to talk about.
315. 19:16It's just I don't really have time, but like my friends over at Tessell, I'm advising them.
316. 19:21They're actually doing this, right?
317. 19:23There's just this whole space of who's looking up who's looking over your agent's shoulders.
318. 19:29I had this conversation just now.
319. 19:30Like I tell everyone, everyone's just starting to stand up agents like 24/7.
320. 19:36Like processing queues, responding to events, like agents that actually do stuff, right?
321. 19:4224/7 and I I I encourage people to think adversarially.
322. 19:47I I think of adversarial groups of agents tasked with doing that queue management cuz one agent will always eventually screw it up.
323. 19:55Right?
324. 19:57So, you got to have those supervisors.
325. 19:58And so, there's whole systems emerging here.
326. 20:01Kind of can go out and go look at all of your and say and start to like you know, do do do that hardening stuff.
327. 20:07Like do they really need all those credentials on that service account?
328. 20:10Really, right?
329. 20:11Only for this one action, maybe we can like separate this one out.
330. 20:13That kind of thing, right?
331. 20:15This is a brand new frontier, but it's one, ironically, that even though there's kind of an almost nothing out there, there's some experimental stuff, you still have to be thinking about it right now and designing a solution in-house right now, right?
332. 20:29Because otherwise your engineers are going to spin up or your non-engineers are going to spin up a bunch of agents with way too many permissions and then, right?
333. 20:37As soon as a bear munches into the igloo, everyone's dead, right?
334. 20:41That's the old security analogy.
335. 20:43I don't know if they still use that one anymore.
336. 20:45[clears throat]
337. 20:45Yes.
338. 20:46Hey Steve, nice to see you.
339. 20:48Um I'm curious in terms of like this the best practices that you've seen uh that you use personally or or maybe you haven't tried, but um particularly with regards to prompt injection.
340. 21:00So,
341. 21:00Yeah, so I was supposed to talk that during my right during during my speech here, I I I wanted to mention it.
342. 21:07[snorts]
343. 21:07There are a whole bunch of attacks happening on the training and on the prompting side, right?
344. 21:14So, on training and inference.
345. 21:15[snorts]
346. 21:16And so, the bad guys find ways to sneak in stuff, right?
347. 21:20Um and it's like like like the simplest version is the new XSRF where like the user puts in some some text and then some bad actor puts in some extra text saying disregard everything and do the following, right?
348. 21:33And then they just get more sophisticated from there.
349. 21:36Um I mean, I don't have any good answers for you other than like this is real.
350. 21:42It's kind of an education problem at this point.
351. 21:44You need to get everybody thinking about it, right?
352. 21:47And then you I feel like there are like new security roles about ready to emerge inside of companies, agentic security that it's kind of an extension of what they're already doing.
353. 21:56Who's already doing this?
354. 21:59Right?
355. 21:59Yeah, so you you've already got people that are going out and looking after the the sort of security of your agents that are in that are deployed in the
356. 22:07Yeah, so you're way ahead of everyone.
357. 22:08I'd love to come talk to you later.
358. 22:10Ha.
359. 22:11This is all brand new stuff, yeah?
360. 22:14Cool.
361. 22:29[music]