WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=f3o0-9Dlw3E NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/f3o0-9Dlw3E.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:12.400 --> 00:00:12.719 Great. s2 00:00:12.719 --> 00:00:14.240 Hi everyone. s3 00:00:14.240 --> 00:00:17.440 We're going to talk today about continuous offensive security. s4 00:00:17.440 --> 00:00:17.680 Okay. s5 00:00:17.680 --> 00:00:19.119 I'm going to speak quite soon. s6 00:00:19.119 --> 00:00:20.400 What does it mean? s7 00:00:20.400 --> 00:00:22.960 Uh but before that, a little bit about me. s8 00:00:22.960 --> 00:00:24.480 I'm very excited being here today. s9 00:00:24.480 --> 00:00:25.600 I'm Ellie. s10 00:00:25.600 --> 00:00:29.599 Uh used to be co-founder and CEO of Helios, a runtime security company. s11 00:00:29.599 --> 00:00:39.440 got acquired by sneak and in the last two and a half years I had the privilege in working in sneak primarily on product management for AI products but more recently s12 00:00:39.440 --> 00:00:49.520 founding field CTO meaning I'm working with developers and with security folks to better understand and to help them shape their strategy around AI security. s13 00:00:49.520 --> 00:00:55.600 So how do you take your AI agents and making sure they are um secured? s14 00:00:55.600 --> 00:00:58.879 How do you work with your agents in production? s15 00:00:58.879 --> 00:01:03.199 Um, back there, can you set up the timer for me, please? s16 00:01:03.199 --> 00:01:04.400 The timer. s17 00:01:04.400 --> 00:01:06.560 So, I'll know the time. s18 00:01:06.560 --> 00:01:07.040 Great. s19 00:01:07.040 --> 00:01:08.880 Okay, now we have the timer also. s20 00:01:08.880 --> 00:01:09.680 Great. s21 00:01:09.680 --> 00:01:12.479 Um, so just so I'll get to know a bit better. s22 00:01:12.479 --> 00:01:13.920 Please raise your hand. s23 00:01:13.920 --> 00:01:17.759 Who is here more from the development side of the table? s24 00:01:17.759 --> 00:01:19.600 Developers. s25 00:01:19.600 --> 00:01:21.119 Okay, great. s26 00:01:21.119 --> 00:01:22.960 From the security side of the table. s27 00:01:22.960 --> 00:01:24.799 Yeah, I already know you. s28 00:01:24.799 --> 00:01:25.360 Great. s29 00:01:25.360 --> 00:01:31.759 and other people like sales, marketing, other people running agents in production that are not from developmental security. s30 00:01:31.759 --> 00:01:32.079 Great. s31 00:01:32.079 --> 00:01:32.640 Okay. s32 00:01:32.640 --> 00:01:34.000 So, we have everyone here. s33 00:01:34.000 --> 00:01:37.840 So, I'm going to tailor this uh session towards everyone. s34 00:01:37.840 --> 00:01:41.759 So, obviously this is an AI engineering conference, right? s35 00:01:41.759 --> 00:01:49.439 So, we already know that the amount of code generated today is incredibly bigger than it used to be before. s36 00:01:49.439 --> 00:01:49.680 Okay. s37 00:01:49.680 --> 00:01:58.799 According to our data, it's 218% more lines of code per the developer than it used to be before. s38 00:01:58.799 --> 00:01:59.119 Okay. s39 00:01:59.119 --> 00:02:06.079 And more than 85% of the developers are actually using coding agent now probably running here today. s40 00:02:06.079 --> 00:02:13.840 It's more than 100% right because I guess that each one of you probably have five different session of cloud code running simultaneously. s41 00:02:13.840 --> 00:02:16.160 Otherwise, you wouldn't be here today. s42 00:02:16.160 --> 00:02:24.480 And the funny thing about those agents running in production and generating fork for us is that it's far less secured than we are used to. s43 00:02:24.480 --> 00:02:32.080 Okay, so 62% of the output generated by LLMs is not secure or it's broken. s44 00:02:32.080 --> 00:02:34.959 Okay, so just imagine this data combined together. s45 00:02:34.959 --> 00:02:43.280 It's meaning that we are generating more code than we ever did and this insecure code is getting into production. s46 00:02:43.280 --> 00:02:56.239 So that's a huge challenge for us as a community and what it actually means is that our backlog of security issues is becoming longer and bigger and much more challenging s47 00:02:56.239 --> 00:02:57.599 to remediate it. s48 00:02:57.599 --> 00:02:57.920 Okay. s49 00:02:57.920 --> 00:03:05.280 So we are actually creating many more issues at a pace that we simply couldn't remediate or mitigate. s50 00:03:05.280 --> 00:03:10.000 And not only that, AI is helping our attackers also. s51 00:03:10.000 --> 00:03:21.920 So they are able to chain low vulnerabilities together and to form critical vulnerabilities from those um low low severity vulnerabilities. s52 00:03:21.920 --> 00:03:33.440 And what's actually is going on is that our attackers are leveraging those LLMs and they're attacking us especially where it's very difficult using um application context. s53 00:03:33.440 --> 00:03:41.599 This is actually where our LLMs are struggled the most because they are creating many more vulnerabilities around those areas. s54 00:03:41.599 --> 00:03:48.159 So obviously we are leveraging AI but our attackers are also leveraging frontier models. s55 00:03:48.159 --> 00:03:48.400 Okay. s56 00:03:48.400 --> 00:03:50.640 And the numbers are simply incredible. s57 00:03:50.640 --> 00:03:50.959 Okay. s58 00:03:50.959 --> 00:04:03.439 We are familiar with an operating model that actually was able to identify more than 600 vulnerabilities uh to breach 600 firewalls in more than 555 countries. s59 00:04:03.439 --> 00:04:06.480 Okay, this is simply unbelievable. s60 00:04:06.480 --> 00:04:13.920 And the time for an AI attack to be effective is constantly getting faster and faster. s61 00:04:13.920 --> 00:04:21.120 Okay, so the average time for an AI attack to be successful is 24 34 minutes. s62 00:04:21.120 --> 00:04:25.120 Okay, well the fastest time is 4 minutes. s63 00:04:25.120 --> 00:04:38.800 Okay, it means that since the moment I began talking, there is probably an AI attack out there that was able to successfully breach a production system and 43% of all the MCP servers s64 00:04:38.800 --> 00:04:40.639 actually has vulnerabilities. s65 00:04:40.639 --> 00:04:48.240 Okay, so combining all of this data together, what we realize is that we are producing much more code than ever. s66 00:04:48.240 --> 00:04:53.600 Each one of us has a few sessions simultaneously of cloud code of codex. s67 00:04:53.600 --> 00:04:56.000 Those this code is insecure. s68 00:04:56.000 --> 00:05:09.120 It's getting into production and the people we are facing the attackers are actually leveraging the same tools and they are acting faster than ever in trying to actually leverage those vulnerabilities s69 00:05:09.120 --> 00:05:13.600 and our backlog is simply getting piled up and we are not able to get it. s70 00:05:13.600 --> 00:05:16.240 And this is exactly what I want to talk about today. s71 00:05:16.240 --> 00:05:21.120 what we should be doing to to help improve our security posture. s72 00:05:21.120 --> 00:05:31.039 And this is ties heavily to how we can leverage offensive security not just for the attackers but for us as defenders. s73 00:05:33.759 --> 00:05:36.720 So why traditional security can't really keep up? s74 00:05:36.720 --> 00:05:42.160 So there is a mixed audience today both people from the development side also people from the security side. s75 00:05:42.160 --> 00:05:47.120 So I want to cover first a few basic concepts okay to make sure everyone is familiar with. s76 00:05:47.120 --> 00:05:53.360 So what used to be really easy about operating was static code scanning okay SAS. s77 00:05:53.360 --> 00:05:56.560 So imagine that you are pushing new code to production. s78 00:05:56.560 --> 00:06:02.720 There is a static scanner that is scanning that and is looking for SQL injection cross-ite scripting. s79 00:06:02.720 --> 00:06:08.240 So it used to be relatively cheap and relatively easy to run this across every code change. s80 00:06:08.240 --> 00:06:09.280 And this is great. s81 00:06:09.280 --> 00:06:13.919 Now what it doesn't catch, it doesn't catch things that are going into runtime. s82 00:06:13.919 --> 00:06:23.680 Okay, so if you have any authorization issues, if you have any configuration issues, for that you will need dynamic application security testing. s83 00:06:23.680 --> 00:06:26.800 Okay, you'll need something that test application dynamically. s84 00:06:26.800 --> 00:06:40.720 One example is bola uh broken object level authorization which basically it means that user A can access the data of user B. Okay, those kind of vulnerabilities you can never find s85 00:06:40.720 --> 00:06:42.319 with static analysis. s86 00:06:42.319 --> 00:06:45.039 For that you need to prop the application in runtime. s87 00:06:45.039 --> 00:06:56.400 So what actually a dust scanner is doing it just takes a predefined set of tons of payloads and it just sending to the application to the different API endpoints s88 00:06:56.400 --> 00:07:00.319 and by that it managed to find vulnerabilities. s89 00:07:00.319 --> 00:07:11.360 So this was also something that could find new things but it wasn't good with the biggest challenge we have with the code that LM are producing and this is the business logic. s90 00:07:11.360 --> 00:07:16.720 So for that we've been you what we've been using as an industry it's the pentesting. s91 00:07:16.720 --> 00:07:16.960 Okay. s92 00:07:16.960 --> 00:07:19.120 So what's the idea about pentesting? s93 00:07:19.120 --> 00:07:30.479 It's a human pentester that really understands the concept and the business logic of the application and is trying to break it and to find all the vulnerabilities. s94 00:07:30.479 --> 00:07:37.599 Now it is the golden standard in application security testing but it has some challenges right. s95 00:07:37.599 --> 00:07:39.599 First of all, it's cost a lot of money, right? s96 00:07:39.599 --> 00:07:49.759 Because you actually needed to have a security researcher, a human, not a human in the loop, a human that is actually doing the testing and trying to break the application. s97 00:07:49.759 --> 00:07:51.919 And that's cost a lot of money. s98 00:07:51.919 --> 00:08:02.960 Now, the second problem is because it cost a lot of money and because the process was so slow, companies can only do it once a year, two times a year, s99 00:08:02.960 --> 00:08:04.319 maybe three times a year. s100 00:08:04.319 --> 00:08:05.120 Okay? s101 00:08:05.120 --> 00:08:16.639 So combining the fact that nowadays our AI attackers are trying to continuously attack us like doing this pan testing two times a year is simply not good enough. s102 00:08:16.639 --> 00:08:16.879 Okay. s103 00:08:16.879 --> 00:08:22.720 So what you going to do with the rest of the 350 days you have a year that no one tested your application. s104 00:08:22.720 --> 00:08:32.880 Not now not only that the results you got the result you got this crazy and glorified report in PDF that you had to send to your developers. s105 00:08:32.880 --> 00:08:39.039 there to take it back to understand what's real and what's not and then to start fixing that and then you had to retest. s106 00:08:39.039 --> 00:08:41.279 Okay, it sounds like a movie from the 90s, right? s107 00:08:41.279 --> 00:08:44.080 It doesn't make sense to do this anymore. s108 00:08:44.080 --> 00:08:45.680 But this was the Gund standard. s109 00:08:45.680 --> 00:08:47.519 This is what we were doing. s110 00:08:47.519 --> 00:08:56.320 Now the thing is this is the best method out there to actually test your application for its business logic. s111 00:08:56.320 --> 00:08:56.800 Okay. s112 00:08:56.800 --> 00:08:58.959 So what changes today? s113 00:08:58.959 --> 00:09:11.760 Today we have a new technology and this is exactly what I want to talk how we can leverage this new area of AI tools and LLM and frontier models to actually do pentesting s114 00:09:11.760 --> 00:09:21.519 on a continuous manner in a scalable manner and this is why we're introducing EVO continuous offensive security. s115 00:09:21.519 --> 00:09:29.040 Okay, so this is obviously very tied up to what we're doing within SNIK, but what I'm going to tell you today is general. s116 00:09:29.040 --> 00:09:37.680 Okay, you can take the principles of what I'm telling you and you can actually verify that with any vendor out there in the expo. s117 00:09:37.680 --> 00:09:41.200 Okay, so I want to I want you to think about the following things. s118 00:09:41.200 --> 00:09:48.000 Okay, so what you need to do today, you need to prepare your production system from the unknowns unknowns. s119 00:09:48.000 --> 00:09:50.480 Okay, and for that you need a new paradigm. s120 00:09:50.480 --> 00:09:50.800 Okay. s121 00:09:50.800 --> 00:09:56.800 So first of all obviously you need the AI pentesting capabilities which I will double click on soon again. s122 00:09:56.800 --> 00:09:59.920 But you also need to start taking the concept of red teaming. s123 00:09:59.920 --> 00:10:00.160 Okay. s124 00:10:00.160 --> 00:10:04.720 So agent red teaming the idea is to take multi-steps attacks. s125 00:10:04.720 --> 00:10:05.120 Okay. s126 00:10:05.120 --> 00:10:15.839 So to try to simulate prompt ejection or data excfiltration and goal aenting and obviously to combine it with dynamic testing that is good at identifying authorization issues. s127 00:10:15.839 --> 00:10:18.959 So we are taking all of those capabilities together. s128 00:10:18.959 --> 00:10:24.240 We are building one suite out of that for offensive security as part of our EVO offering. s129 00:10:24.240 --> 00:10:38.000 And the idea is to help you actually mitigate or help you uh handle the fact that the attackers are leveraging the same tools that we are using and they are doing that in a continuous manner s130 00:10:38.000 --> 00:10:45.839 24/7 finding vulnerabilities faster than before and your backlog simply cannot can stand up with that. s131 00:10:45.839 --> 00:10:49.839 So what we're doing differently within AI pentesting. s132 00:10:49.839 --> 00:10:50.079 Okay. s133 00:10:50.079 --> 00:11:00.320 So we I told you before the great promise of pentesting was always the fact that the human and the end really understand the business context. s134 00:11:00.320 --> 00:11:00.640 Okay. s135 00:11:00.640 --> 00:11:07.839 So now we are taking this to the LLM and if there is something that LLM are very good at is really trying to understand the business context. s136 00:11:07.839 --> 00:11:12.560 So behind the scene we are leveraging the LLMs and we are doing that in a continuous manner. s137 00:11:12.560 --> 00:11:12.959 Right? s138 00:11:12.959 --> 00:11:19.040 So no longer you're going to do things like do pent testing twice a year or three times a year. s139 00:11:19.040 --> 00:11:21.600 We're going to do it on every code change. s140 00:11:21.600 --> 00:11:22.000 Okay? s141 00:11:22.000 --> 00:11:31.680 Every PR that you send, every delta, we're going to run this pentest for you and we're going to tell your developers what do they need to fix and how do they need to do it differently. s142 00:11:31.680 --> 00:11:38.880 Obviously we leverage our understanding of the application context, but we also help reduce the false positive. s143 00:11:38.880 --> 00:11:42.800 For us as a industry, we have a severe problem of false positive. s144 00:11:42.800 --> 00:11:52.079 So we are actually proving you that the vulnerabilities that is that we are finding are actually exploitable and we just just flag them as bugs. s145 00:11:52.079 --> 00:12:05.360 We actually help you prove that they are exploitable and we take contexts for many different machines and for many different environments and we combine them together to help guide our LLMs. s146 00:12:05.360 --> 00:12:20.160 Okay, so this is what we're doing differently and I think this is really what's makes this solution so unique and so well positioned to the fact that each one of us have a few coding gadgets running in production and pushing more code than ever. s147 00:12:20.160 --> 00:12:21.519 Now how do we do it? s148 00:12:21.519 --> 00:12:23.920 How does it work behind the scenes? s149 00:12:23.920 --> 00:12:32.800 Okay, so what I want you to understand is we are not taking the static scanners or the dust scanners and just combine him with LLM. s150 00:12:32.800 --> 00:12:36.320 Okay, we build it from the ground up using LLM. s151 00:12:36.320 --> 00:12:40.240 So we have a series of a few agents working together. s152 00:12:40.240 --> 00:12:42.480 The first one is the LLM orchestrator. s153 00:12:42.480 --> 00:12:45.120 Okay, imagine it as the brain. s154 00:12:45.120 --> 00:12:53.920 This agent is working according to a dynamic plan that it follows and it knows how to operate all the different sub agents. s155 00:12:53.920 --> 00:12:57.680 The first agent or actually the second agent is the recon. s156 00:12:57.680 --> 00:13:01.920 Okay, you can imagine it as your elite intelligence unit. s157 00:13:01.920 --> 00:13:17.440 The idea is to collect all the data it can about the different network endpoints, the different APIs, the different fingerprints, collect all of this data together, formalize an architecture out of that and hand it to the next agent. s158 00:13:17.440 --> 00:13:20.240 And the next agent is the vulnerability testing. s159 00:13:20.240 --> 00:13:24.560 It's actually again a series of a few specialized agents. s160 00:13:24.560 --> 00:13:28.320 Some of them are targeting the well-known classes of vulnerabilities. s161 00:13:28.320 --> 00:13:39.040 Some of them are actually hunting for vulnerabilities in the wild and they are trying to break your application to understand the business context and to understand where can attacker can get from s162 00:13:39.040 --> 00:13:44.880 and then they hand it into the judge as I like to call it the exploit validation. s163 00:13:44.880 --> 00:14:01.360 So actually every vulnerability that the previous agent find are now being handed over to this judge and the goal of the judge is to tell us whether this is a real vulnerability or or not to tell us whether this can be exploitable or not. s164 00:14:01.360 --> 00:14:04.480 Exactly because we want to reduce the cognitive load. s165 00:14:04.480 --> 00:14:10.240 Exactly because we want to reduce the amount of false positive we have in the system. s166 00:14:10.240 --> 00:14:15.839 Coming up next is the remediation agent because everything we do we want to make actionable. s167 00:14:15.839 --> 00:14:22.800 Okay, we don't want to report about a vulnerability that eventually the developers they don't know what they can do with that. s168 00:14:22.800 --> 00:14:29.519 And coming after that is the reporting agent because everything needs to be reported. s169 00:14:31.199 --> 00:14:43.199 So what makes us different and those are the point I urge you to check with every vendor out there trying to provide you with the solution or whether you want to implement a solution inhouse for AI pentesting. s170 00:14:43.199 --> 00:14:46.639 This is a huge industry right now. s171 00:14:47.600 --> 00:14:56.480 So I think the most important thing you need to remember is and you already know that especially the developers in the room context is everything. s172 00:14:56.480 --> 00:15:00.959 your LLM are as good as the context that you feed them with. s173 00:15:00.959 --> 00:15:07.839 So what we're trying to do, we are not just taking our AI pen testing solution as a standalone. s174 00:15:07.839 --> 00:15:11.440 We actually feed it with data from all the previous scanners. s175 00:15:11.440 --> 00:15:24.399 Okay, so the the statics code scanner, the dynamic code scanner, all of the different agent engines are being combined together and they are being fed into a our AI pentesting solution s176 00:15:24.399 --> 00:15:33.199 because that way we know how to tailor and we know how to guide the LLM where to look for and as you know also for mythos or for frame s177 00:15:33.199 --> 00:15:39.680 when those models know where to look the results are significantly better. s178 00:15:39.680 --> 00:15:45.360 So that's one thing we're doing that is you know very unique and very differentiated. s179 00:15:45.360 --> 00:15:51.120 Um and also what I want you to remember is that one solution is not enough. s180 00:15:51.120 --> 00:15:51.440 Okay. s181 00:15:51.440 --> 00:16:01.360 The great thing about it is when you combine those things together and based on the data that we see both from the labs and from real data from customers s182 00:16:01.360 --> 00:16:08.639 combining the data together is really what's important and you can't really you can't really use just one of them. s183 00:16:08.639 --> 00:16:08.959 Okay. s184 00:16:08.959 --> 00:16:12.240 There are certain classes of vulnerabilities. s185 00:16:12.240 --> 00:16:14.639 This dust is the best scanner for that. s186 00:16:14.639 --> 00:16:15.199 Okay? s187 00:16:15.199 --> 00:16:24.399 Because you have a set of really exhaustive pre um a set of really exhaustive uh payloads that you can send to your application. s188 00:16:24.399 --> 00:16:27.120 So it's very good for example with quite scripting. s189 00:16:27.120 --> 00:16:27.440 Okay. s190 00:16:27.440 --> 00:16:31.920 So dust for crossite scripting is the best solution. s191 00:16:31.920 --> 00:16:39.920 But if you want again like business logic then you should combine it with uh pentesting because it will be the best solution. s192 00:16:39.920 --> 00:16:49.920 So I think what I really want you to take from this, if there is one thing I want you to take from here is to realize that the world is changing. s193 00:16:49.920 --> 00:16:50.959 You already know that. s194 00:16:50.959 --> 00:16:52.880 That's what made you come in. s195 00:16:52.880 --> 00:17:07.919 But you know we used to test our application once every few months because the change and the velocity of the change was so slow and we're we had people at the end that were reasoning about the application s196 00:17:07.919 --> 00:17:09.120 and about the change. s197 00:17:09.120 --> 00:17:11.760 But that world is does not longer exist. s198 00:17:11.760 --> 00:17:12.640 Right? s199 00:17:12.640 --> 00:17:23.120 Right now we are pushing code to production 24/7 and we have LLMs that are trying to reason about the application context and they are trying to break it continuously. s200 00:17:23.120 --> 00:17:26.319 Some of them can do it in like as fast as four minutes. s201 00:17:26.319 --> 00:17:30.480 Okay, that's almost five time of the lecture of the session that I just did. s202 00:17:30.480 --> 00:17:36.880 So when you're coming to look for new solutions for this problem, you have to ask yourself four questions. s203 00:17:36.880 --> 00:17:37.520 Okay. s204 00:17:37.520 --> 00:17:46.000 First of all, do we run this solution in a continuous way or is it just a point solution that we run every once in a while? s205 00:17:46.000 --> 00:17:48.080 Okay, that's a major difference. s206 00:17:48.080 --> 00:17:52.720 Second, do they really reason about the application context? s207 00:17:52.720 --> 00:18:01.120 Okay, because given the nature of those LLM, application context is really where the new vulnerabilities are coming from. s208 00:18:01.120 --> 00:18:06.400 And third, what more context can they digest? s209 00:18:06.400 --> 00:18:08.799 Okay, how can they enrich the data? s210 00:18:08.799 --> 00:18:15.600 Because the richer the data is, the better and by the way cheaper the LLM operation is going to be. s211 00:18:15.600 --> 00:18:27.520 And the fourth part is do they really help me prove that there is a bug and to make sure that this is exploitable or they they just flag it as a bug. s212 00:18:27.520 --> 00:18:39.120 Okay, so our goal is not only to help you find the bugs faster, but to actually do it better and cheaper and to help you as a developer or as the security people here s213 00:18:39.120 --> 00:18:42.000 focus on what matters most. s214 00:18:42.000 --> 00:18:45.840 Um, so thank you very much and uh I used to do it in every lecture. s215 00:18:45.840 --> 00:18:50.640 You know, my my my dream was to be a rockstar, but I had to be a cyber entrepreneur. s216 00:18:50.640 --> 00:18:52.720 So instead of that, I'm going to take pictures. s217 00:18:52.720 --> 00:18:55.440 So hands up everyone. s218 00:18:56.960 --> 00:18:58.320 Okay, great. s219 00:18:58.320 --> 00:19:00.880 Thank you so much. s220 00:19:01.440 --> 00:19:02.559 And I'm here for questions. s221 00:19:02.559 --> 00:19:04.720 If you want, go for the boot. s222 00:19:04.720 --> 00:19:05.760 It's very close here. s223 00:19:05.760 --> 00:19:06.799 sneak.io. s224 00:19:06.799 --> 00:19:09.919 Meet Evo, our new AI security offering. s225 00:19:09.919 --> 00:19:12.480 Thank you so much.