WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=wEc9aG7cRQc

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/wEc9aG7cRQc.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:02.920 --> 00:00:04.040
Hello.

s2
00:00:04.040 --> 00:00:13.800
This session is about why do your AI agents disagree with itself and what to do about it.

s3
00:00:13.800 --> 00:00:18.440
And this is a Dyan Huang Lin, and you can call me Dyan.

s4
00:00:19.720 --> 00:00:22.640
A little bit about my background.

s5
00:00:22.640 --> 00:00:31.400
My journey with AGI start from my PhD at Imperial College London, where I was working on continual learning.

s6
00:00:31.400 --> 00:00:41.320
Later, I worked with Professor Josh Tenenbaum at MIT on one-shot learning, meaning learning from one single examples.

s7
00:00:42.600 --> 00:00:46.480
Afterwards, I was lucky that Alexa was launching.

s8
00:00:46.480 --> 00:00:53.120
I was the first three appliance scientists on the question answering team at the Alexa.

s9
00:00:53.520 --> 00:01:01.360
Later, attracted by a startup working on AGI at the Silicon Valley called Vicarious.

s10
00:01:01.360 --> 00:01:05.600
So, today Vicarious is part of the Google DeepMind.

s11
00:01:05.600 --> 00:01:12.720
And there I got to work on the exciting frontier research on zero-shot transfer learning.

s12
00:01:13.000 --> 00:01:25.200
After working on AGI for a few years, I decided to pivot to the applied world at Zscaler, which is cybersecurity company.

s13
00:01:25.520 --> 00:01:33.320
There I got to apply a variety of different machine learning model to adjust the cybersecurity challenge.

s14
00:01:34.240 --> 00:01:44.200
There for 5 years, uh I decided to co-found my own company because of pinpoint I have seen at the Zscaler.

s15
00:01:44.480 --> 00:01:54.480
So, company is building AI agent to auto-triage your security alerts and which make your SOC more efficient.

s16
00:01:55.280 --> 00:02:02.480
Uh we are super proud that Chrome innate has been acquired by Data Dog earlier this year.

s17
00:02:02.480 --> 00:02:10.520
So now I'm part of the Data Dog and leading the development of self-evolved agent.

s18
00:02:14.520 --> 00:02:21.400
So today I'm going to share with you the problem you're probably seeing your day-to-day building AI agent.

s19
00:02:21.400 --> 00:02:23.880
The inconsistency.

s20
00:02:24.680 --> 00:02:32.000
Then we are discussed about where this inconsistency come from in order to figure out the solutions.

s21
00:02:32.880 --> 00:02:43.320
And we will discuss a few trade-off among the different solutions and then eventually we'll show you the experimental results to show how feasible they are.

s22
00:02:46.040 --> 00:02:48.200
You'll probably see this.

s23
00:02:48.200 --> 00:02:51.000
Same model, same input.

s24
00:02:51.000 --> 00:02:52.720
But different output.

s25
00:02:52.720 --> 00:02:55.840
Here I don't mean the wording different.

s26
00:02:55.840 --> 00:03:00.000
I mean semantically different output.

s27
00:03:01.680 --> 00:03:06.959
You might be thinking that's the stochastic nature of LM.

s28
00:03:07.680 --> 00:03:09.239
Hold on to that thought.

s29
00:03:09.239 --> 00:03:12.640
Let's look at a few concrete examples.

s30
00:03:13.640 --> 00:03:25.160
So here imagine you're giving a task for sentiment analysis where we're supposed to label each of the review of the hotel either positive or negative.

s31
00:03:26.400 --> 00:03:29.400
This is typical NLP task.

s32
00:03:29.400 --> 00:03:33.160
And today's LM is really good at this.

s33
00:03:33.160 --> 00:03:36.480
However, there's one problem.

s34
00:03:36.840 --> 00:03:49.200
When you pipe this input to the same LM model, occasionally you will see different verdict coming out of it when you run multiple times.

s35
00:03:49.320 --> 00:03:55.000
That means a single evaluation run didn't tell you the whole story.

s36
00:03:55.000 --> 00:04:09.800
You need to repeat your evaluation multiple times in order to get a holistic picture by the average over the different runs results.

s37
00:04:10.400 --> 00:04:21.359
And this sounds like a technical inconvenience, but the reality is is way more problematic than just inconvenience.

s38
00:04:21.359 --> 00:04:26.160
Let me show you an example in a different domain in cybersecurity.

s39
00:04:26.680 --> 00:04:40.680
Imagine you are a security analyst where your day-to-day job is triage the security alerts and decide whether it's malicious that you need to take certain actions to stop the attack

s40
00:04:40.680 --> 00:04:47.000
or it's false alarm, meaning it's actually benign, which you could ignore.

s41
00:04:47.600 --> 00:04:56.840
So, here example alert you got is there's a failed logging attempt to Gmail account detected from suspicious IP.

s42
00:04:58.680 --> 00:05:13.040
And if we give such similar task to this one to AI agent and run it different times, you will see sometimes it stay consistently benign, sometimes it stay consistently suspicious

s43
00:05:13.040 --> 00:05:15.240
across different runs.

s44
00:05:15.240 --> 00:05:22.880
However, there are cases where it flip-flop between benign and suspicious.

s45
00:05:23.960 --> 00:05:33.840
Now, your customer will have a difficulty using such product because they will wonder, "Which one should I trust?"

s46
00:05:33.840 --> 00:05:41.640
So, the inconsistency is causing a trust issue in your product.

s47
00:05:41.880 --> 00:05:59.880
Imagine if you were in a POC bake-off, one vendor give the consistent wording all the time while the other vendor have this flip-flop wording.

s48
00:05:59.880 --> 00:06:03.960
You can imagine which one would win the deal.

s49
00:06:03.960 --> 00:06:08.520
Now, I'm pretty sure you want to solve this problem.

s50
00:06:11.920 --> 00:06:18.320
First, we have we have to figure out where the inconsistency come from.

s51
00:06:21.240 --> 00:06:33.919
The good news is the data points tend to flip-flop actually concentrate around the decision boundary, the so-called gray zone.

s52
00:06:34.040 --> 00:06:37.520
Let me illustrate with the earlier example.

s53
00:06:37.520 --> 00:06:42.320
Here in the sentiment analysis, we have two reviews.

s54
00:06:42.320 --> 00:06:53.960
The first one actually no matter how many times you run through the LM model, it will always give you the positive label.

s55
00:06:53.960 --> 00:06:59.760
In this case, it's pretty obvious is the review is super positive.

s56
00:06:59.760 --> 00:07:05.680
However, the second case is the one tend to flip.

s57
00:07:05.720 --> 00:07:09.640
If we look closer, it's kind of on the boundary.

s58
00:07:09.640 --> 00:07:19.080
Sometimes there's certain word is kind of a relative positive, but sometimes there's a little bit negative.

s59
00:07:20.160 --> 00:07:29.400
In this case, you'll find that even human experts will have a disagreement on these cases.

s60
00:07:29.400 --> 00:07:32.800
In fact, there's no right or wrong answer.

s61
00:07:32.800 --> 00:07:37.880
It really depends on particular company's policy and their preference.

s62
00:07:37.880 --> 00:07:49.680
For hotel thinking that the such review is not something I can do to improve the memorial kind of experience, memorable experience.

s63
00:07:49.680 --> 00:07:55.160
Then, that mean not to label as naughty because there's nothing I can do.

s64
00:07:55.160 --> 00:08:01.080
But some hotel might care about it and want to label them as natives so they can improve.

s65
00:08:01.080 --> 00:08:06.240
So, that comes down your preference.

s66
00:08:06.600 --> 00:08:10.480
Let's look the other cybersecurity example.

s67
00:08:10.720 --> 00:08:14.880
Here is another case on the boundary.

s68
00:08:14.880 --> 00:08:21.960
And why it's in the boundary is here there's a few attempts.

s69
00:08:21.960 --> 00:08:26.000
Indeed, there's attacker knocking on the door.

s70
00:08:26.000 --> 00:08:32.080
But attacker might not getting yet.

s71
00:08:32.640 --> 00:08:42.159
So, for enterprise customers who always have attacker knocking on the door, well, if they haven't getting in the environment, they don't want to worry about it.

s72
00:08:42.159 --> 00:08:45.360
Otherwise, they will be flooded by this type of alerts.

s73
00:08:45.360 --> 00:08:51.560
There's nothing they need to do because they are blocked outside already.

s74
00:08:51.920 --> 00:09:04.520
However, if the attacker do manage to get the correct password and pass MFA and do manage getting the environment, then it become very seriously and you need to take certain actions.

s75
00:09:04.520 --> 00:09:14.600
So, almost similarly at the beginning behavior, but very different outcome and different response needed.

s76
00:09:14.600 --> 00:09:20.480
Depends on whether it's truly attack or attacker is still outside.

s77
00:09:20.480 --> 00:09:33.000
So, here whether we label them malicious or benign depends on company preference whether they wanted to be notified about such situation.

s78
00:09:33.600 --> 00:09:41.000
So, again, this is the example where it depends on your preference.

s79
00:09:41.040 --> 00:09:48.800
And additional information to disambiguate between the two different scenarios.

s80
00:09:50.400 --> 00:10:00.480
So from these two very different domains, you will see that actually the data points tend to flip-flop.

s81
00:10:00.480 --> 00:10:13.400
Are the one close to the decision boundary where it need your clarity to disambiguate which side it should really belongs to.

s82
00:10:14.320 --> 00:10:19.760
So these data points are where human experts tend to make a mistake.

s83
00:10:19.760 --> 00:10:25.160
And the traditional machine learning classifier will also struggle with.

s84
00:10:25.160 --> 00:10:30.960
So in other words it's not your AI agent's fault.

s85
00:10:30.960 --> 00:10:38.040
Your AI agent simply point out the ambiguity that already exists.

s86
00:10:38.720 --> 00:10:47.760
So now we know where the root cause of the inconsistent is.

s87
00:10:47.760 --> 00:10:53.320
The next step is identify where exactly they are.

s88
00:10:53.320 --> 00:10:54.560
Who are they?

s89
00:10:54.560 --> 00:10:57.400
And fix them.

s90
00:10:59.760 --> 00:11:09.440
Surprisingly and not surprised the solution to identify the gray zone actually exists already in machine learning.

s91
00:11:09.440 --> 00:11:13.160
And it's called active learning.

s92
00:11:13.240 --> 00:11:21.200
So the idea of active learning is you have a lot of data point to be labeled.

s93
00:11:21.200 --> 00:11:28.760
Imagine you'd have your initial model and you put it in production, maybe in the monitor mode.

s94
00:11:28.839 --> 00:11:32.360
And you wanted to check the quality.

s95
00:11:32.360 --> 00:11:35.280
But you don't have bandwidth to check all of them.

s96
00:11:35.280 --> 00:11:39.640
Otherwise, it lose a point of having agent to do the work.

s97
00:11:39.640 --> 00:11:46.200
But you still want to identify where the model didn't do well.

s98
00:11:46.200 --> 00:11:56.480
And then active learning is to find a way to select the data points where tend to make mistake.

s99
00:11:56.480 --> 00:12:01.880
And that's where it tells you where to pull your attention.

s100
00:12:03.280 --> 00:12:16.520
So let's go over the traditional um machine learning models active learning pipeline and see how is different in the case of the new LM or AI agent error.

s101
00:12:16.560 --> 00:12:24.720
So first, you train your initial model and then you make predictions on the unlabeled data set.

s102
00:12:24.720 --> 00:12:29.440
It could be the production online data.

s103
00:12:29.440 --> 00:12:42.600
And then from there, you see there's certain cases where the model wasn't sure because the probability is close to 2.5. And those data points are the high informational one

s104
00:12:42.600 --> 00:12:46.600
that your model is going to learn the most.

s105
00:12:46.600 --> 00:12:53.839
The uncertainty is a type of the signal to tell you where the model tend to make a mistake.

s106
00:12:53.839 --> 00:13:08.120
Another popular type of way to figure out the problematic one is query by committee, meaning that it by a few different models or even same model multiple times and then you identify

s107
00:13:08.120 --> 00:13:10.240
the disagreement.

s108
00:13:10.240 --> 00:13:21.440
And that's where the model tend to make a mistake as well and that's where you need human clarification on label so the model can learn it from.

s109
00:13:23.080 --> 00:13:37.680
The after you surface these problematic data points and for human to take a second look to make sure the label is actually correct or do we doesn't need additional information to dis

s110
00:13:37.680 --> 00:13:41.840
ambiguate those cases meaning adding new features.

s111
00:13:41.840 --> 00:13:48.320
And then you can add those label data set or potentially additional features to the next round of training.

s112
00:13:48.320 --> 00:13:54.640
Then you retrain the model and then proceed uh continue this cycle.

s113
00:13:54.640 --> 00:14:08.240
So then you can see in this particular active learning cycle you spend very little uh amount or at least in other way it's very efficient way to identify the problematic

s114
00:14:08.240 --> 00:14:17.400
data points and spend your attention on those point where your model tend to learn the most.

s115
00:14:18.360 --> 00:14:20.880
So that's the traditional one.

s116
00:14:20.880 --> 00:14:27.520
The good news is the LM way is not very different.

s117
00:14:27.520 --> 00:14:30.680
There's a major two part which we need some refinement.

s118
00:14:30.680 --> 00:14:34.160
One is on this selection strategy.

s119
00:14:34.160 --> 00:14:37.200
The other part is about retraining.

s120
00:14:37.960 --> 00:14:47.800
On the selection strategy one um here we suggest you use the one about identify disagreement.

s121
00:14:47.800 --> 00:14:54.960
At least from what we have tried so far the uncertainty score from LM is not very reliable.

s122
00:14:54.960 --> 00:14:58.160
It's kind of a LM model doesn't know what it doesn't know.

s123
00:14:58.160 --> 00:15:04.480
So when it's very confident it doesn't mean that it was the it would is correct.

s124
00:15:04.480 --> 00:15:22.440
So that However, the the disagreement from different runs or from different models actually give you a more reliable signal where the model actually not sure about its verdict and it need human guidance.

s125
00:15:25.760 --> 00:15:37.360
And then after you identify this group or the uh discriminant and need attention, you label them and provide a feedback like what you did before.

s126
00:15:37.360 --> 00:15:42.240
Then afterwards, you might be thinking, "Oh, time to retrain the model."

s127
00:15:42.240 --> 00:15:50.480
Yes or no. Yes, one option is retrain the model, but fine-tuning model is expensive.

s128
00:15:50.480 --> 00:15:57.560
Here, we are proposing a more lighter weight solution, easier to iterate.

s129
00:16:01.280 --> 00:16:07.120
It's about augment your agent with semantic and episodic memory.

s130
00:16:21.360 --> 00:16:25.200
Let's use the previous example to illustrate what I mean here.

s131
00:16:25.200 --> 00:16:38.360
Um as we talked about, in this case, the attempt to flip, it's mainly missing the information about you a particular company policy or preference.

s132
00:16:39.680 --> 00:16:45.200
Here, we can add it to the um the knowledge base.

s133
00:16:45.200 --> 00:16:56.000
If this customer complain about something outside hotel's control, then classify as positive because you don't want to pay attention to this.

s134
00:17:00.160 --> 00:17:16.120
And then in the cybersecurity example, something similar, you can put in in that the password spray, meaning that the attacker trying to guess your password and try it with different times, even fail, and continuously.

s135
00:17:16.120 --> 00:17:19.680
So, if without successful logging, then that should be benign.

s136
00:17:19.680 --> 00:17:26.920
But, in contrast, if it succeed with final logging, then that should be malicious.

s137
00:17:26.920 --> 00:17:35.880
So, here you actually identify the additional information to help disambiguate the two and then label them separately.

s138
00:17:35.880 --> 00:17:41.520
And that's a clarity that help your model to learn.

s139
00:17:41.520 --> 00:17:47.720
Also, help your human experts to label more consistently.

s140
00:17:48.280 --> 00:17:53.040
So, this type of domain knowledge sharpen your decision boundary.

s141
00:17:53.040 --> 00:17:55.880
And also, help the human beings.

s142
00:17:55.880 --> 00:18:02.000
And this type of knowledge is like factual knowledge, is part of actually belong to your semantic memory.

s143
00:18:02.000 --> 00:18:05.880
And this is in contrast with the episodic memory.

s144
00:18:05.880 --> 00:18:21.360
Episodic memory is in the case where you haven't get around to distill the reasoning behind the earlier case why it should be on malicious or the benign side.

s145
00:18:21.360 --> 00:18:27.800
Instead, you said that I have seen these similar cases before and it was labeled as benign.

s146
00:18:27.800 --> 00:18:33.440
And let me leverage those past similar cases and they made a decision accordingly.

s147
00:18:33.440 --> 00:18:38.600
And the advantage of the this episodic memory way is it's relatively automatically.

s148
00:18:38.600 --> 00:18:40.840
It has less human intervention.

s149
00:18:40.840 --> 00:18:54.200
I mean, the human intervention was done in the past already when during when you're doing a job online, um you can reference them, at least your AI agent can reference them automatically.

s150
00:18:54.200 --> 00:19:02.040
They can making decisions without waiting for you to distill the reasoning and putting in the semantic knowledge.

s151
00:19:02.040 --> 00:19:13.600
So, the idea is that the identify a similar case to the one under examination and then leverage their past decision as a reference.

s152
00:19:13.600 --> 00:19:23.720
And then imagine there's a new case which is coming in the middle, doesn't belong to any of past group, and that's the one you still need a human attention.

s153
00:19:23.720 --> 00:19:33.040
And this is how also um where episodic memory resolve recurring case automatically.

s154
00:19:33.040 --> 00:19:43.360
And this particular useful in the case of cyber security alerts because a lot of noise especially those false positive noise are the one keep recurring.

s155
00:19:43.360 --> 00:19:48.280
And that's the one are the low hanging fruit you can automate away.

s156
00:19:48.280 --> 00:20:00.240
And with that human review which is a precious attention and bandwidth you have, now you can concentrate on those where episodic memory failed to address.

s157
00:20:00.240 --> 00:20:07.160
And this is also how we chose between semantic memory and episodic memory.

s158
00:20:07.160 --> 00:20:10.080
The two are actually not contradicting.

s159
00:20:10.080 --> 00:20:12.200
In fact, they are complementary.

s160
00:20:12.200 --> 00:20:18.520
Basically, you use episodic memory to address those recurring situations automatically.

s161
00:20:18.520 --> 00:20:28.720
And then for what remains still reflecting after the reference from the past or no reference to be able to use, then you pass on to human review.

s162
00:20:28.720 --> 00:20:36.440
And then human review distill the domain knowledge and put it in the semantic memory to be used next time.

s163
00:20:37.880 --> 00:20:43.440
So here we talk about a solution actually brings three benefits.

s164
00:20:43.440 --> 00:21:02.400
First, we identify these data point near the decision boundary and use the domain knowledge in semantic memory and this past similar case in the episodic memory, we're able to improve the the consistency

s165
00:21:02.400 --> 00:21:04.240
significantly.

s166
00:21:04.240 --> 00:21:11.320
So, now you have a more trustworthy AI agent for your user to use.

s167
00:21:11.320 --> 00:21:19.680
And two other byproduct of this solution is you have a very efficient quality control.

s168
00:21:19.680 --> 00:21:36.440
You don't need to check every agent output, but instead a subset of them, which your algorithm uh active learning think is potentially problematic, and you check them, either provide additional

s169
00:21:36.440 --> 00:21:43.520
information or clarify the label, and your model will learn from there.

s170
00:21:44.000 --> 00:21:56.160
So, most importantly, your labeling effort is very small, but high return.

s171
00:21:56.480 --> 00:22:03.720
Last but not least, now you are gathering your customer feedback along this way.

s172
00:22:03.720 --> 00:22:11.920
You not only have a trustworthy consistent model and high efficacy one, you also have one adapt to your customer environment.

s173
00:22:11.920 --> 00:22:21.280
I'm pretty sure your customer will love your product if you your AI agent listen to them and adapt to them.

s174
00:22:21.920 --> 00:22:26.560
So, here are some experiment results from uh real data points.

s175
00:22:26.560 --> 00:22:35.080
So, here we collect uh 93 alerts, cybersecurity alerts, and then we run them three times.

s176
00:22:35.080 --> 00:22:43.040
If without the solution we are proposing here, a quarter of them will flip-flop the verdict.

s177
00:22:43.040 --> 00:22:55.200
In contrast, after applying our solution or using episodic memory, about 15% of them become consistent.

s178
00:22:55.200 --> 00:22:59.800
However, there's still 10% of them remaining inconsistent.

s179
00:22:59.800 --> 00:23:02.360
They still flip.

s180
00:23:02.600 --> 00:23:08.240
Sometimes because there's no reference, similar cases in past paper used.

s181
00:23:08.240 --> 00:23:15.600
Sometimes even there's a reference to be used, they still disagree after the second thought.

s182
00:23:15.600 --> 00:23:16.920
But not a problem.

s183
00:23:16.920 --> 00:23:23.320
The episodic memory do the automatic lock down all the inconsistency for 15%.

s184
00:23:23.320 --> 00:23:33.960
The remaining 10% human review and provide the additional knowledge to disambiguate those tricky cases.

s185
00:23:34.000 --> 00:23:43.000
After that, you also adapt to the customer environment that you want to learn.

s186
00:23:46.760 --> 00:24:00.440
So, the three takeaways that I hope you have after this talk is number one, inconsistency isn't usually your model problem.

s187
00:24:00.440 --> 00:24:14.400
Stop blaming your model, but instead focus your energy on label issue and potentially insufficient information and help your AI agent with this additional clarity.

s188
00:24:15.480 --> 00:24:22.520
Second, the model disagreement isn't a bug, but feature.

s189
00:24:22.520 --> 00:24:28.960
Treat each disagreement as an opportunity for your model to learn.

s190
00:24:29.040 --> 00:24:34.040
Third, fine-tuning isn't your only option.

s191
00:24:34.040 --> 00:24:39.520
Augment your agent with semantic and episodic memory.

s192
00:24:39.840 --> 00:24:48.640
I hope you enjoy seeing your model keep improving and adapting to your customer environment.

s193
00:24:50.760 --> 00:24:58.160
Last but not least, I want to thanks to my colleagues and wonderful friends at Data Dog.

s194
00:24:58.160 --> 00:25:13.720
Special thanks to Ditun and Harsh, Anusha, and Stefan, Saeed, and Mati to give me a lot of enjoyable discussions and helped me to clarify some of the um the concept.

s195
00:25:14.040 --> 00:25:22.240
And also, thank you for everyone in the Beats Secured Analyst team to make this journey enjoyable.

s196
00:25:22.640 --> 00:25:24.600
And thank you for listening.

s197
00:25:24.600 --> 00:25:30.800
Hope you will enjoy training your model and having it improving.

s198
00:25:31.640 --> 00:25:32.560
Thank you.

s199
00:25:32.560 --> 00:25:34.560
Bye.