WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=O-CBZ3JtRvo

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/O-CBZ3JtRvo.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:12.880 --> 00:00:13.920
Hello everyone.

s2
00:00:13.920 --> 00:00:17.199
Okay, thanks for showing up at this data quality.

s3
00:00:17.199 --> 00:00:21.279
So as you saw, we probably talk a little bit about other things than data quality.

s4
00:00:21.279 --> 00:00:30.080
and uh and first I actually tell you about why I'm very excited about this talk and why actually I accepted to uh to come present it with Fury.

s5
00:00:30.080 --> 00:00:45.520
Um there's two reason for that and the first reason is I think and what you will see today is that cyber security is a much wider field for AI a much wider playing field and exploration field than you might think.

s6
00:00:45.520 --> 00:01:00.879
And in particular what we'll show you today is uh a benchmark that arithmetic and Yuri has been developing and I've been doing a little bit advising which I think is even close to things like arc AGI 3 for people who has been following progress around AGI

s7
00:01:00.879 --> 00:01:12.960
in which that um by which I mean that this benchmark ask for people oh yeah cool to do if you have been playing with RKGI3 some of you who knows here RKGI3

s8
00:01:12.960 --> 00:01:17.680
or Arcadia in general one person.

s9
00:01:17.680 --> 00:01:20.560
Okay, we're in very data quality uh field.

s10
00:01:20.560 --> 00:01:24.479
Basically, this ask models to try to understand what's happening in the world.

s11
00:01:24.479 --> 00:01:35.280
And it's actually small games that the models need to understand basically what's the current state, how we can play with it and how we can actually change the state of the game.

s12
00:01:35.280 --> 00:01:38.640
So, uh it's it's actually something you can play yourself.

s13
00:01:38.640 --> 00:01:48.399
So basically it has a it ask a model to understand when you click somewhere something happening at another place and you may think this is very simple and that we should be passed that

s14
00:01:48.399 --> 00:02:06.159
on our way to AGI and the the thing you will discover if you play with this benchmark is no like models have one to two% success rate on on this generic benchmark and the reason is the current model even though they're really good they build they can't really build a dynamic model of what's happening in the world

s15
00:02:06.159 --> 00:02:21.280
or what's happening in any type of world and I think the the benchmark that arithmetic has been developed is a benchmark that's also extremely challenging for model in that they need to understand what's happening and to act accordingly to the world model they've been building on the fly

s16
00:02:21.280 --> 00:02:33.680
so that's the first reason I think this benchmark is really interesting and why I'm actually uh very happy to show you that and the second reason is u I think there's a lot of things that uh open source model can bring in cyber security

s17
00:02:33.680 --> 00:02:45.040
and we tend to have this very binary view of closour model are good for cyber open source model are bad and I think what we want to show today is that uh open source model are one part of the solution

s18
00:02:45.040 --> 00:02:56.560
to uh cyber security challenges today and in particular if you think in terms of attack and defense and how it will be in the future this balance uh we think that cyber

s19
00:02:56.560 --> 00:03:04.640
uh and open source model used in cyber security will be key to actually be able to to to uh solve the defense solution.

s20
00:03:04.640 --> 00:03:12.560
So there is a future I think where cyber is alive and every everyone is well protected and I'm pretty sure this future involve open source model.

s21
00:03:12.560 --> 00:03:16.159
Now Yuri is also kind of an impressive uh impressive person.

s22
00:03:16.159 --> 00:03:18.000
I'm really happy I met him.

s23
00:03:18.000 --> 00:03:28.080
He was starting at Harvard dropped to like build this idea of what the future of cyber should be and it's an honor to have you on stage with me.

s24
00:03:28.080 --> 00:03:44.080
Thank you so much Thomas and thank you everyone who came and I'm really grateful and I'm also grateful for the work we've done together to build this benchmark which is uh I think incredibly difficult for the models and actually shows some really really interesting leaps in places that I think we still have way to go.

s25
00:03:44.080 --> 00:03:52.720
Um I guess the way we've been thinking about the problem and the reason we set out to do this is it's very clear that the economics of cyber are fundamentally shifting.

s26
00:03:52.720 --> 00:04:00.000
There's this um inherent thing that that is inherent to cyber which is that attackers need to choose their resource really wisely.

s27
00:04:00.000 --> 00:04:07.920
And if you sort of think about cyber as as a house in a way then my job is to block every door and close every window and make sure that there's no way in.

s28
00:04:07.920 --> 00:04:12.000
And the attacker's job is to find at least one seam, one crack, one thing I missed.

s29
00:04:12.000 --> 00:04:16.400
And then once inside, my job is to put sensors and anything I can to keep them out.

s30
00:04:16.400 --> 00:04:29.199
Um, and the whole stack, the entire world of cyber that we've been building for the past 20ish years has been based on this economics that the attackers have to choose their targets and we do everything we can across it to protect ourselves.

s31
00:04:29.199 --> 00:04:31.759
It is true that that is changing in really dramatic ways.

s32
00:04:31.759 --> 00:04:34.080
The models are incredibly powerful.

s33
00:04:34.080 --> 00:04:36.560
Um, they're able to find a ton of primitives.

s34
00:04:36.560 --> 00:04:38.400
They're able to find a bunch of zero day exploits.

s35
00:04:38.400 --> 00:04:39.040
We're seeing this.

s36
00:04:39.040 --> 00:04:42.000
There's so many news and chaos around this point.

s37
00:04:42.000 --> 00:04:49.680
Um, and on the other hand, it seems like we as defenders don't seem to be prepared for for the this world and the way that it's coming.

s38
00:04:49.680 --> 00:04:54.240
And I um actually think that I might be on the the wrong one.

s39
00:04:54.240 --> 00:05:07.360
What I want to show is sort of this idea of um so so if you think about the way cyber has been, this is definitely I think the way we've been thinking about AI and cyber for the past many many many

s40
00:05:07.360 --> 00:05:08.160
uh months.

s41
00:05:08.160 --> 00:05:10.800
And it's it's freaky and it's getting really scary.

s42
00:05:10.800 --> 00:05:12.560
And this is my point on the house.

s43
00:05:12.560 --> 00:05:23.199
And part of the reason we think this is happening is if cyber is this game of skill and speed, then a very skilled attacker using something like mythos can now choose a bunch of targets all at once.

s44
00:05:23.199 --> 00:05:26.479
That is a a real reality that is quite scary.

s45
00:05:26.479 --> 00:05:31.759
And the truth is the picture really doesn't change that much when we move to a really strong open source model as well.

s46
00:05:31.759 --> 00:05:37.440
So the question is what do we do in this world where the economics of cyber offense are shifting so much?

s47
00:05:37.440 --> 00:05:42.960
Um, and part of the problem with the existing stack is that defensive systems have to operate at scale.

s48
00:05:42.960 --> 00:05:46.560
And that means that we have always very limited human intervention.

s49
00:05:46.560 --> 00:05:49.680
And so we're sort of bound by what the models can do out of the box.

s50
00:05:49.680 --> 00:05:58.880
So if we live in this world where the models are becoming so powerful, advancing so fast, we think uh like Thomas said, the solution also has to be the models themselves.

s51
00:05:58.880 --> 00:06:00.960
Um, this is our first solution.

s52
00:06:00.960 --> 00:06:03.919
We can just all go live in the woods and lock ourselves out.

s53
00:06:03.919 --> 00:06:09.600
Um, another solution is to figure out how we get the models to be much much much more capable.

s54
00:06:09.600 --> 00:06:18.080
We have no doubt that open source models have to be part of this solution because um they allow for many things that we'll talk about as well and there's this deep need for collaboration.

s55
00:06:18.080 --> 00:06:26.639
And so what I want to show you today and I see the the clock ticking on me is um sort of this theory that if we've already done this before for coding,

s56
00:06:26.639 --> 00:06:28.319
we can do this for cyber.

s57
00:06:28.319 --> 00:06:33.600
There's a reason to be optimistic, which is really controversial in the context of AI and cyber lately.

s58
00:06:33.600 --> 00:06:34.880
Um, we've done this before.

s59
00:06:34.880 --> 00:06:43.039
I think if you go back a year, it was very clear that all of coding was going to be transformed thanks to um the models getting better and better and better.

s60
00:06:43.039 --> 00:06:48.560
And we're sort of seeing the early innings of that with cyber right now where everyone's talking about cyber and everyone's freaking out.

s61
00:06:48.560 --> 00:06:59.919
But what if through very high quality evals, very high quality data, good benchmarks, we could get to a place where the attackers are um simply outperformed by very very very good defenders.

s62
00:06:59.919 --> 00:07:09.120
And so our goal in arithmetic is to be able to get the models to be really capable at cyber security to the point where we can rebuild this new stack

s63
00:07:09.120 --> 00:07:12.560
um that's based on the models uh winning the models on the other side.

s64
00:07:12.560 --> 00:07:14.240
So I'm really excited to show you Masov.

s65
00:07:14.240 --> 00:07:16.479
It's our first benchmark that we're releasing.

s66
00:07:16.479 --> 00:07:22.240
Um, our first fundamental idea is that we can't capture all of cyber in one singular benchmark.

s67
00:07:22.240 --> 00:07:26.240
That's a bit like saying that swimming, an F1 driver, and a basketball player is the same thing.

s68
00:07:26.240 --> 00:07:27.680
It doesn't work.

s69
00:07:27.680 --> 00:07:30.960
Um, and so we focus specifically on access control.

s70
00:07:30.960 --> 00:07:32.800
Really quickly, why access control?

s71
00:07:32.800 --> 00:07:37.599
Um, it's sort of the first door to any target in cyber begins with my ability to get a foothold.

s72
00:07:37.599 --> 00:07:42.479
So, if you think about an attacker on the one hand, they're trying to get to some privileged thing.

s73
00:07:42.479 --> 00:07:48.479
uh if Thomas and I are working on the same company, I'm some ML engineer, he's an admin, uh what are the things?

s74
00:07:48.479 --> 00:07:51.360
Can I find a way to do things that I'm not allowed to do?

s75
00:07:51.360 --> 00:08:01.919
Uh in my current privileged position, it actually leads to being the number one on the OAS list and has created this sort of $30 billion industry and for years these are number one vulnerabilities.

s76
00:08:01.919 --> 00:08:05.199
The reason they exist and this is to the RKGI point.

s77
00:08:05.199 --> 00:08:06.720
These are logic based vulnerabilities.

s78
00:08:06.720 --> 00:08:10.160
So it's not just about bugs in the code that I find and I need to patch.

s79
00:08:10.160 --> 00:08:21.680
It's about very very very big systems and somewhere between them there's these logic breaks where it's possible that one thing checks for something specific in the code another checks for something else and that sort of leads to everything breaking

s80
00:08:21.680 --> 00:08:28.160
and so what we're trying to do is we're trying to figure out how we get the models to really reason very very hard and not just do pattern matching

s81
00:08:28.160 --> 00:08:44.159
that's where a lot of works goes into the data quality and our data is created first by humans I think that's really important right now to find out of distribution things we need humans to go do the search our team is all uh based on very deep vulnerability researchers and nerds who love to hack who are trying

s82
00:08:44.159 --> 00:08:47.600
to get really really good at cyber at AI capabilities.

s83
00:08:47.600 --> 00:08:51.440
So we find our own zero days in widely uh distributed open source software.

s84
00:08:51.440 --> 00:09:06.560
We use that to create these t these real live huge environments of a bunch of different um applications chained together that then allows us to basically create this blackbox setting where the model doesn't see the code and it doesn't know about the zero date because we found it ourselves

s85
00:09:06.560 --> 00:09:13.760
and it has to find a way uh to reason across this entire surface and understand exactly what the exploitation is.

s86
00:09:13.760 --> 00:09:21.360
And so to do that, we don't give it access to the internet or the codebase, but we do give it sort of all the basic tooling it would need to be able to execute a task well.

s87
00:09:21.360 --> 00:09:25.279
And everything because the tasks are so difficult, everything has a deterministic greater.

s88
00:09:25.279 --> 00:09:34.000
And so across the entire exploitation and the discovery chain, every single step can be deterministically verified, allowing us to see how deep it got within the chain.

s89
00:09:34.000 --> 00:09:41.680
And finally, this is credit to Eugene from Entropic who we slightly stole this graphic from, but it really does capture really well the way we've set up our eval

s90
00:09:41.680 --> 00:09:46.080
where basically you have uh inputs on the one hand based on a real zero days.

s91
00:09:46.080 --> 00:09:51.519
The agent so we're thinking about as the model plus its harness and some blackbox tooling that has to find.

s92
00:09:51.519 --> 00:09:54.000
Then we have a verifiable grader which is a binary pass.

s93
00:09:54.000 --> 00:09:58.640
Was the model able to do something it wasn't allowed to do as an underprivileged user?

s94
00:09:58.640 --> 00:10:02.399
And then we have our deterministic grading in every step along the way.

s95
00:10:02.399 --> 00:10:07.120
So what I want to show you now is an illustration for all the security buffs in the room.

s96
00:10:07.120 --> 00:10:08.959
It's an illustration guys.

s97
00:10:08.959 --> 00:10:19.040
Um but the idea is how a real solve looks and fundamentally what we have here is a real uh task of ours where it's a chain between keycloak vault and a broker

s98
00:10:19.040 --> 00:10:21.279
and I start as a very low privileged user.

s99
00:10:21.279 --> 00:10:23.760
I need to figure out how to get to production code.

s100
00:10:23.760 --> 00:10:25.040
So what we're going to see is a solve.

s101
00:10:25.040 --> 00:10:28.640
Each one of our tasks has a solve script of what a real solution looks like.

s102
00:10:28.640 --> 00:10:36.560
There's a real zero day that we found that we submitted for verification to the maintainers where um there's a check whether I'm an admin or not.

s103
00:10:36.560 --> 00:10:37.760
It only checks by name.

s104
00:10:37.760 --> 00:10:40.959
And another aspect of this checks whether by ID.

s105
00:10:40.959 --> 00:10:51.200
So that allows me as a user to change the name of um the real admin inherent their um their privilege and then use that to escalate myself.

s106
00:10:51.200 --> 00:10:57.279
And what I'm really trying to illustrate is sort of this longchain 16st step type of logic that the model has to do.

s107
00:10:57.279 --> 00:11:04.800
And if it's not able to understand inherently the system, it's way too b wide for it to test everything sort of shoot across the space.

s108
00:11:04.800 --> 00:11:11.040
So what we're going to see now is a real attempt by uh GPT 5.5 and then opus as well trying to solve this task.

s109
00:11:11.040 --> 00:11:18.160
And what you're going to see is a sort of chaotic trying everything, jumping between everything, probing a bunch of different stuff.

s110
00:11:18.160 --> 00:11:27.519
It does even reach the check, but it never makes the logical leap that it's supposed to be able to change the admin's own permission, the own name in order to bypass this permissioning.

s111
00:11:27.519 --> 00:11:29.120
So, this is exactly what we're trying to test.

s112
00:11:29.120 --> 00:11:34.560
Can the model understand uh leaps, logic leaps that are inherent to the system?

s113
00:11:34.560 --> 00:11:41.440
H a really important point just like RKGI, everything you do in a live system in permissioning changes other stuff in the system.

s114
00:11:41.440 --> 00:11:46.880
So, the model needs to be able to hold this model of the world it's living in and iterate through it.

s115
00:11:46.880 --> 00:11:50.720
Um and then fundamentally at the end of this it writes code.

s116
00:11:50.720 --> 00:11:58.720
It writes exploitation code and it needs to be able to do um to reason really really thinly and understand exactly what the exploitation is in order to be able to execute.

s117
00:11:58.720 --> 00:12:03.279
And we can really see the difference between models that have succeeded some of the tests and models that haven't.

s118
00:12:03.279 --> 00:12:10.160
And then now we're going to do something that I've been told to never do, which is show a live demo of a real system on stage.

s119
00:12:10.160 --> 00:12:14.399
And so let's hope um let's hope we don't get uh there.

s120
00:12:14.399 --> 00:12:16.079
And I I know we're basically out of time.

s121
00:12:16.079 --> 00:12:18.240
So what we're seeing here is what's called Bach.

s122
00:12:18.240 --> 00:12:20.000
It's our internal system because it's an orchestrator.

s123
00:12:20.000 --> 00:12:22.160
This is how we run our actual eval.

s124
00:12:22.160 --> 00:12:25.680
Um what we see is the actual results of the benchmark.

s125
00:12:25.680 --> 00:12:27.440
The benchmark right now is incredibly hard.

s126
00:12:27.440 --> 00:12:29.519
There's only one solve at K1.

s127
00:12:29.519 --> 00:12:36.639
Um and then at K5 there is um it remains only GPT and the public models is is able to solve this.

s128
00:12:36.639 --> 00:12:45.440
That's why the partial graders are so critical to be able to really see what the model is able to do and what and how deep within exploitation chain they can get.

s129
00:12:45.440 --> 00:12:53.519
I'm going to load quickly the sort of the way we think about these environments which is because we're looking for performance over time.

s130
00:12:53.519 --> 00:12:56.639
We really measure how capable is the model at making specific leaps.

s131
00:12:56.639 --> 00:13:02.480
And so what you'll see is a results of exploitation on specific one of our environments.

s132
00:13:02.480 --> 00:13:09.120
And then if we zoom in then you can really see how sort of GPT 5.5 is the only model that's able to make this leap.

s133
00:13:09.120 --> 00:13:12.240
The model other models sort of have been able to reason across everything.

s134
00:13:12.240 --> 00:13:21.120
If we look at the discovery phase they do capture nearly all the different information they need and they never are able to make the leap into what is the exploitation they need to do.

s135
00:13:21.120 --> 00:13:23.920
This is exactly the type of capability that we believe.

s136
00:13:23.920 --> 00:13:34.880
If every model in the world could get really really really good at doing this and very fast, that should give a lasting defense uh and capability to the defenders that the attackers simply don't have right now.

s137
00:13:34.880 --> 00:13:43.920
Um and then finally, I think the way we sort of reason through these and work through them um is is quite cool and I want to show you.

s138
00:13:43.920 --> 00:13:56.079
So again, if we go into one of our tasks called fall fall time, we can really see sort of the way we spend our days, which is really really really trying to understand what are the specific failure modes a model does.

s139
00:13:56.079 --> 00:13:59.920
You can see that this is longer ryzen, not because it's waiting for code to run.

s140
00:13:59.920 --> 00:14:04.399
It's constantly working over three hours and it's still been unable to solve the tasks.

s141
00:14:04.399 --> 00:14:11.120
And then I guess the way I spend all my day is is quite literally going through all the traces of what the models did, why, and how.

s142
00:14:11.120 --> 00:14:21.040
Um, and yeah, and I think my I think the final point I want to make and I'll pass it back to Thomas is sort of next what if we can

s143
00:14:21.040 --> 00:14:34.800
uh not just have this chart which is really cool sort of have this chart um of a really super cool future model that's very very fast in its understanding of what the capability leaps need to be

s144
00:14:34.800 --> 00:14:43.680
and I really believe that with everything happening now it is critical that we get cyber capabilities to the point where um we can defend much much faster.

s145
00:14:43.680 --> 00:14:46.720
The only way to replace the old stack is through the models.

s146
00:14:46.720 --> 00:14:48.320
Um, and I'll pass it back to Thomas.

s147
00:14:48.320 --> 00:14:57.920
I think also the only way to do that is through a real array of strong open source models and collaboration that we can post train on and that we can post train to each network and to each environment as well.

s148
00:14:57.920 --> 00:14:58.160
Yeah.

s149
00:14:58.160 --> 00:15:04.560
So, as you saw this benchmark is quite different from mythos type of uh we read the code and we find the vulnerabilities.

s150
00:15:04.560 --> 00:15:17.120
Here basically the model is operating in a real environment where you know there's a authentification place somewhere doesn't know what's there then there's another like network like um like microservices

s151
00:15:17.120 --> 00:15:26.480
you need to access and use and it has basically zero information of that and it all start like you show by a zero zero day vulnerability that we have so everything is new

s152
00:15:26.480 --> 00:15:36.480
so I think there's there's a lot of research uh to be done and understand how models can actually understand and work on that and so the First step is getting some good data.

s153
00:15:36.480 --> 00:15:40.880
So the idea is to have some good benchmarks starting have some good data on how this is operating.

s154
00:15:40.880 --> 00:15:49.519
I think the second step is being able to fine-tune models and try to understand how we can pro protect against that right and and the big challenge here is going to be speed.

s155
00:15:49.519 --> 00:15:54.399
So you already told that several times going to be the speed of attacker versus defense, right?

s156
00:15:54.399 --> 00:15:58.720
When they start to enter, you have to be able to see what's happening and catch them.

s157
00:15:58.720 --> 00:16:06.560
and speed will be where you know you you want to have a specialized model that's maybe running on specialized hardware and actually is is going to be very important

s158
00:16:06.560 --> 00:16:18.320
and here I think the the danger is to say we're just going to rely on two company that everyone knows here to solve all of that for us I think the solution is just to take our future and say well it's going to be a speed challenge

s159
00:16:18.320 --> 00:16:28.399
we're going to train our model we're going to run them fast and make them available to basically every company who wants to be protected so exciting I would say it's uh as everything in cyber security.

s160
00:16:28.399 --> 00:16:38.639
It's both very interesting but also a big challenge and and u well something you you have to not mess up I would say.

s161
00:16:39.519 --> 00:16:40.160
Awesome.

s162
00:16:40.160 --> 00:16:43.680
Um yeah thank you very much.

s163
00:16:43.680 --> 00:16:52.880
Um for anyone that wants to collaborate work on this work we have a forum uh we are going to uh work with people on this data directly.

s164
00:16:52.880 --> 00:16:54.160
We'd love to hear from you.

s165
00:16:54.160 --> 00:17:01.120
anyone who's really passionate about any other field in cyber, like Thomas said, we have to do this across sort of not just access control, a bunch of different things.

s166
00:17:01.120 --> 00:17:07.199
And the only way this is going to work is through a lot of um a lot of post- training data and really really capable models.

s167
00:17:07.199 --> 00:17:08.880
Congrats for your first presentation.

s168
00:17:08.880 --> 00:17:09.671
Thanks.

s169
00:17:09.671 --> 00:17:11.671
[laughter]

s170
00:17:24.081 --> 00:17:26.081
[music]