WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=s67bE2Ur3bY

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/s67bE2Ur3bY.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:13.840 --> 00:00:14.639
Okay.

s3
00:00:14.639 --> 00:00:16.240
Good afternoon everyone.

s4
00:00:16.240 --> 00:00:20.720
Uh I'm Sai Krishna and firstly I would like to thank all of you for attending the talk.

s5
00:00:20.720 --> 00:00:23.199
This is the final session of the final day.

s6
00:00:23.199 --> 00:00:27.760
So I realize um thanks you thanks a lot for attending the talk.

s7
00:00:27.760 --> 00:00:29.599
So let's get started.

s8
00:00:29.599 --> 00:00:33.920
This conference has really been about agentic systems, right?

s9
00:00:33.920 --> 00:00:45.440
Every workshop that we attend, the keynotes, the speaker sessions, the conversations that we have been having in the hallways, all of us, all of us have been discussing agents in one form or the other.

s10
00:00:45.440 --> 00:00:50.079
And I would like to declare that you know we have won like we have built the agentic systems.

s11
00:00:50.079 --> 00:00:51.360
We can build them.

s12
00:00:51.360 --> 00:00:55.120
For you know a very basic reference what is an agent?

s13
00:00:55.120 --> 00:01:01.199
An agent is nothing but a combination of systems model which is the brain of the agent.

s14
00:01:01.199 --> 00:01:07.840
We have harnessed some form of orchestration around it and a set of tools that lets the agent do something.

s15
00:01:07.840 --> 00:01:09.600
Now this is pretty simplistic.

s16
00:01:09.600 --> 00:01:12.000
We can build an agent in an afternoon.

s17
00:01:12.000 --> 00:01:15.600
It'll start doing useful things by the evening.

s18
00:01:15.600 --> 00:01:22.720
And yes, there are a lot of issues today in terms of you know the traces etc. But we can build systems today.

s19
00:01:22.720 --> 00:01:27.119
And I would like to project forward trying to see what comes after this.

s20
00:01:27.119 --> 00:01:42.560
And let me start by a basic observation which is that almost every agent we build today has the customer of size one clims of them all of them cater to one person.

s21
00:01:42.560 --> 00:01:51.840
Even the programming assistants that we build even when they are deployed in an enterprise setting, they are typically geared towards one customer.

s22
00:01:51.840 --> 00:01:53.200
So this is great.

s23
00:01:53.200 --> 00:01:55.840
We know the problems and challenges with this setup.

s24
00:01:55.840 --> 00:01:58.960
We know the shape of the things that we want to build.

s25
00:01:58.960 --> 00:02:04.880
But probably the next agent we are going to build is not going to have one single customer.

s26
00:02:04.880 --> 00:02:08.080
It's probably going to be serving a group.

s27
00:02:08.080 --> 00:02:14.080
On top of that, it's probably going to be invariable such as glasses and it's going to be on all day.

s28
00:02:14.080 --> 00:02:25.280
So, one of my advisers used to say, you know, you work hard as an engineer to solve a problem and then the moment you solve it, you realize that the question itself has slightly changed.

s29
00:02:25.280 --> 00:02:38.560
Maybe we are heading towards a setting like that where all of our engineering we have built it for a agentic systems targeting one customer whereas we might be entering an era where we are deploying these agents in group settings

s30
00:02:38.560 --> 00:02:45.840
and group settings pose uniquely different challenges compared to settings where we have single users.

s31
00:02:45.840 --> 00:02:58.959
So I have been working uh inspired by this um with an agent called Judith which is deployed in a group setting among friends and family for a period of 8 months.

s32
00:02:58.959 --> 00:03:06.159
Let me illustrate what I mean in the talk by taking a few examples uh taken from the production system itself.

s33
00:03:06.159 --> 00:03:23.840
On the left we see one example from this week where it's deployed in a group of attendees of this conference and I was asking it okay how do we go to the conference venue and the agent didn't chose to not answer in the group but DM' me because of the privacy issue.

s34
00:03:23.840 --> 00:03:33.360
Similarly, this is a conversation from a couple of weeks ago where my wife and I we were trying to organize an event and it's trying to sync up all the calendars and you know

s35
00:03:33.360 --> 00:03:37.280
make sure we get a slot which is available for everyone.

s36
00:03:37.280 --> 00:03:39.840
Then there is a proactive aspect as well.

s37
00:03:39.840 --> 00:03:54.080
All of us I think after the uh release of coding agents we are operating on pretty light sleep and the fact that the agent understands uh we are under less sleep and you know we have

s38
00:03:54.080 --> 00:04:04.319
um you know trouble decision-m that's not impressive the impressive part uh when a glasses agent spoke to me this was I was driving it didn't it chose to not announce this

s39
00:04:04.319 --> 00:04:16.799
from the media of the car but it chose to speak it directly to me in the glasses preserving ing the privacy again and then um a group setting has conversations that are lasting over a period of time.

s40
00:04:16.799 --> 00:04:28.720
So this is an example where a bunch of friends have been discussing a particular topic that is constantly evolving and the agent was uh intelligent enough to curate the content and the memory

s41
00:04:28.720 --> 00:04:31.828
which is relevant and filter out the things that are not needed.

s42
00:04:31.828 --> 00:04:32.160
[snorts]

s43
00:04:32.160 --> 00:04:41.120
And finally a simple application from an agent that my daughter uses who is three-year-old who is using the agent to you know learn lot of things like capitals of the countries different

s44
00:04:41.120 --> 00:04:51.520
uh numbers etc. Keeping us posted as well as to the progress of the kid and making sure that she doesn't forget the things that she learned and we also are aware of the things that he's learning.

s45
00:04:51.520 --> 00:05:04.560
So all of these are examples where agents in a group setting have slightly different dimensions when they are deployed and there are the the aspect of group deployment is also not unique.

s46
00:05:04.560 --> 00:05:17.440
There have been attempts in the last 6 to 8 months of trying to deploy agents in a group setting under controlled scenarios try to try and see you know how they fare etc.

s47
00:05:17.440 --> 00:05:23.440
So leading to the recognition of uh the growth of agents in a group setting.

s48
00:05:23.440 --> 00:05:30.320
My talk today is going to be focused mainly on three aspects in group chats agents.

s49
00:05:30.320 --> 00:05:36.560
One the guard of the agent which is what what lets in the memory of the agent.

s50
00:05:36.560 --> 00:05:45.919
Today in agentic systems we know that the harness involves a lot of orchestration in terms of the memory but group charts take it to the next level.

s51
00:05:45.919 --> 00:05:55.120
And then the routing the routing here is not the model routing to save tokens but routing of the information the end information to the final user.

s52
00:05:55.840 --> 00:06:01.280
So let's get started with uh the memory the the security layer which I'm calling jetou.

s53
00:06:01.280 --> 00:06:04.720
It's a mythological character from Indian mythology.

s54
00:06:04.720 --> 00:06:11.840
The tagline from here is that we can't secure an agentic system like we secure a large language model.

s55
00:06:11.840 --> 00:06:20.240
What I mean by that is a large language model when it hallucinates we can inspect the output and design stuff on top of it.

s56
00:06:20.240 --> 00:06:30.960
But an agentic system by nature operates on things as well in addition to the large language model which means the surface area that the agentic system touches is much more richer

s57
00:06:30.960 --> 00:06:37.120
and more vast compared to a large language model and it is amplified in a group setting.

s58
00:06:37.120 --> 00:06:49.759
So imagine this right when we have an agentic system deployed in a even in a personal setting it's going through every web page that uh is available and the attackers

s59
00:06:49.759 --> 00:07:01.840
who can do prompt injection and various other types of attacks which I'm going to talk about they have access to it all day every group message even though the people in the group we have added them but

s60
00:07:01.840 --> 00:07:19.440
imagine a scenario where somebody takes a screenshot of something and uploads it uh every GitHub issue that we post and track updates on there have been lot of cases of late where you know tokens have been exploited that way every email that we get the promotional emails etc the agent reads them and all of these are surface

s61
00:07:19.440 --> 00:07:38.639
areas where the agent can be exposed to uh vulnerabilities and therefore we can't guard everything and remove everything from entering there has to be a balance between how we design the security layer and which is why I'm saying it's different from the security layer when it comes to typical ical large language models.

s62
00:07:38.639 --> 00:07:51.759
So to illustrate this, one of the recent papers which came out is uh highlights a very interesting uh phenomena where the punch line is that we can't read our way or we can't

s63
00:07:51.759 --> 00:08:00.960
model uh check our way to safety and what we see here are two separate skills in the context of a typical large language model.

s64
00:08:00.960 --> 00:08:04.160
One of them is an OCR module.

s65
00:08:04.160 --> 00:08:06.879
The other is a reporting module.

s66
00:08:06.879 --> 00:08:10.800
Both of the skills static scans are pretty good.

s67
00:08:10.800 --> 00:08:11.840
They're pristine.

s68
00:08:11.840 --> 00:08:13.440
They pass them.

s69
00:08:13.440 --> 00:08:22.720
But the paper runtime skill audit found out that a static scan surviving code can break at runtime.

s70
00:08:22.720 --> 00:08:32.959
And the the paper when safe skills collide also identified that two skills which are benign at the surface when they run together they can be malignant.

s71
00:08:32.959 --> 00:08:45.316
For example, uh imagine the skills are sitting in our infrastructure but the the the attacker is attacking the the content that the agent is reading

s72
00:08:45.316 --> 00:08:45.360
[snorts]

s73
00:08:45.360 --> 00:08:56.640
which implies that the OCR extracts the information but the reporting agent when it sends the information along with the information it also sends our PII to a third party

s74
00:08:56.720 --> 00:08:58.800
to put the numbers to the story as well.

s75
00:08:58.800 --> 00:09:06.160
This is not something which happens um at uh at low statistical frequency.

s76
00:09:06.160 --> 00:09:12.240
The the papers have observed that around 90% of the attacks have this.

s77
00:09:12.480 --> 00:09:23.839
So if you want to think about safeguarding our multi- aent multi-group setting architectures from this probably the first level of defense has to come at the boundary.

s78
00:09:23.839 --> 00:09:31.839
Instead of guarding whatever the agent reads, we let the agent read everything and then design a guard which is deterministic.

s79
00:09:31.839 --> 00:09:32.640
So it's fast.

s80
00:09:32.640 --> 00:09:35.200
We don't have issues with respect to latency.

s81
00:09:35.200 --> 00:09:37.360
But design when it is taking the action.

s82
00:09:37.360 --> 00:09:49.120
So for example, when it is reading bash variables or when it is exporting something or when it is reading secret variables with respect to configs etc. And then since we can design a deterministic

s83
00:09:49.120 --> 00:09:59.519
system using traditional NLP or regax based approaches we can flag these out saying hey some of them are benign so let's allow them but some of them need approval

s84
00:09:59.519 --> 00:10:04.480
etc. So this is the three class categorization that we can come up with as a naive approach.

s85
00:10:04.480 --> 00:10:14.880
Now uh we all know that this approach is good but we can probably do slightly better which would be to say can we train a model on top of it as well.

s86
00:10:14.880 --> 00:10:27.279
So within Judith there is a component which tries to look at this data and tries to build a supervised finetuning model an SLM which is Lora fine-tuned to try and predict

s87
00:10:27.279 --> 00:10:39.760
if there is something malignant in the data channel separate the data channel firstly from the instruction channel and only give out that information which is purely benign.

s88
00:10:39.760 --> 00:10:51.360
So the ideal response here would be if the instruction is summarize this page for me and the website that we have crawled contains some prompt injection attack or any other form of attack

s89
00:10:51.360 --> 00:10:57.920
to obscure that information and the model just performs the task as required.

s90
00:10:59.279 --> 00:11:02.959
So uh we have also benchmarked this on inspect agent.

s91
00:11:02.959 --> 00:11:15.120
That's one of the benchmark uh data sets available here and it's it's the naive approach here is basically at 50% which is a coin flip and it improves on top of that.

s92
00:11:15.120 --> 00:11:28.880
A very interesting caveat here is that a particular uh attack was spotted where instead of writing the text normally if you write it interspersed with dots like I do.

s93
00:11:28.880 --> 00:11:30.240
LIi.

s94
00:11:30.240 --> 00:11:35.279
K. The regax based approaches and most of the static approaches fail at that.

s95
00:11:35.279 --> 00:11:42.560
Whereas a learned model uh would probably catch it which is where the attempt to build a learn model comes in.

s96
00:11:42.560 --> 00:11:56.320
But to summarize when we are building a uh security layer for agentic system be it personal or be it group based setting we want to be able to build a system not at the input because that is going to gate everything

s97
00:11:56.320 --> 00:12:10.639
but at the action surface where the agent actually is performing some task and maybe a better approach than a naive approach like regex or traditional NLP is to try to build a model against which we can do some hill climbing

s98
00:12:13.120 --> 00:12:26.560
Once the input comes into the model, the next important interesting thing when it comes to group especially when it comes to group settings would be how do we organize the memory because in a traditional setting in a traditional model

s99
00:12:26.560 --> 00:12:30.639
uh the memory implementations can be very naive.

s100
00:12:30.639 --> 00:12:34.880
A memory implementation can be done just using embeddings or files.

s101
00:12:34.880 --> 00:12:47.360
But the interesting uh thing to note is that the model is just the engine especially when it comes to a group chat when we deploy it in either friends group or in a work setting.

s102
00:12:47.360 --> 00:12:52.880
The memory is what the agent becomes and it's becoming in real time it's evolving in real time.

s103
00:12:52.880 --> 00:13:03.760
So we have to be very careful when it comes to how we store things, how we curate the the thing and how we forget some of the things that are stored in memory.

s104
00:13:03.760 --> 00:13:12.800
It's it has applications with respect to latency because the less we store the better token usage and everything else.

s105
00:13:13.120 --> 00:13:19.360
So I would like to talk about uh five aspects here uh briefly mentioning uh each of them.

s106
00:13:19.360 --> 00:13:21.279
The first one is what do we store?

s107
00:13:21.279 --> 00:13:33.920
Let's take a typical conversation um like that happens in a group chat setting where people are discussing one particular occasion where the first person says you know hey um let's go to this place

s108
00:13:33.920 --> 00:13:45.839
and this is the day at which we are going now there's a follow-up conversation somebody else adds some more information to it which keeps happening now one approach which commonly

s109
00:13:45.839 --> 00:13:52.639
uh naively what we do is to store everything and then keep compacting the conversation as the conversation becomes overhead.

s110
00:13:52.639 --> 00:14:01.680
A probably smarter way would be to ex extract some form of atomic information, atomic bits from this conversation.

s111
00:14:01.680 --> 00:14:08.240
In this case, the atomic bits would be uh anything that is relevant to the conversation which might come into future.

s112
00:14:08.240 --> 00:14:19.839
So once we build a model like that, the important thing here becomes how do we uh design auto rators for this and the auto rators have to be designed on extract

s113
00:14:19.839 --> 00:14:30.399
did we extract high value atomic facts or not given a conversation are these the relevant information or not and then are these hierarchically related to each other or not?

s114
00:14:30.399 --> 00:14:37.360
The temporal aspect as well because some things become more important as the time goes along and some things we might have to drop.

s115
00:14:37.360 --> 00:14:53.199
And then are we retrieving the right set of information given the current query and then is there some form of a intelligent retrieval as opposed to crawling everything by organizing information in form of a graph etc.

s116
00:14:54.240 --> 00:15:01.680
So uh the other interesting aspect when it comes to memory would be what uh what we store and what we forget.

s117
00:15:01.680 --> 00:15:12.639
As the memory becomes bloated and bloated, we might want to train a very simple machine learning model or an SLM to try and figure out what are the important concept context

s118
00:15:12.639 --> 00:15:15.040
uh information that we need to store.

s119
00:15:15.040 --> 00:15:25.600
And a typical approach, a a very simple approach discussed in the paper learning what not to forget is where they learn a relevance scorer which keeps scoring continually.

s120
00:15:25.600 --> 00:15:35.440
The important word here being continually because in a group chat setting or in a group work setting the context keeps evolving and different things become important at different points

s121
00:15:35.440 --> 00:15:47.440
and we can once we have a relevant scorer which is continuously adapting we can save much more on the tokens because now we are doing knowledge based compaction.

s122
00:15:48.320 --> 00:16:07.600
One other thing when it comes to serving these stored memories is uh whether whether we use cloud cloud-based serving or if we use local based serving we need to be aware of the fact that we all of us use KV caches and KV caches break when we try to do something cute with respect to the model.

s123
00:16:07.600 --> 00:16:14.639
Therefore building an injection engine which is a aware of KV caching also helps there.

s124
00:16:16.000 --> 00:16:21.040
So all of these uh are the components which are important while designing a memory.

s125
00:16:21.040 --> 00:16:32.880
Again um the memory component here is slightly different from the memory for my personal engine and all of these become extremely relevant uh in a group setting.

s126
00:16:34.079 --> 00:16:38.959
So I'll talk about the final part of the talk which is the privacy aspect.

s127
00:16:38.959 --> 00:16:51.759
uh a shared assistant is not just a large model which is shared between different people but it's almost a new social contract because the information that is uh passed to a shared agent

s128
00:16:51.759 --> 00:17:10.319
it's either public or private depending upon the context in which it is presented in the same information for example grocery list is benign for everybody but information about salary or information about somebody's health becomes extremely private as well as public information depending upon the context

s129
00:17:10.319 --> 00:17:15.439
the data has not changed the room in which it is deployed has changed.

s130
00:17:15.439 --> 00:17:30.559
So a very neat trick for this would be instead of storing all of the data all the time take inspiration from uh the way human brain works and try to have a common shared memory layer but train different

s131
00:17:30.559 --> 00:17:35.919
uh Laura adapters on top of of each adapter for each of the users.

s132
00:17:35.919 --> 00:17:37.919
Therefore the permissions are baked in.

s133
00:17:37.919 --> 00:17:43.600
Now this is baking the permissions not by using code but by using machine learning itself.

s134
00:17:43.600 --> 00:17:52.400
The user as an engram paper is is a great reference uh of how to do it at scale and also knowing when to speak and when not to speak.

s135
00:17:52.400 --> 00:18:03.600
A typical observation from where agents are deployed in group setting is they tend to o be over proact be over uh articulative when they are not asked questions.

s136
00:18:03.600 --> 00:18:14.960
And a tip and a simple approach here which is currently still in progress would be to train a model to determine uh it could be as simple as a robata classifier but to determine

s137
00:18:14.960 --> 00:18:20.080
the role and when an agent is allowed to speak and not speak.

s138
00:18:20.720 --> 00:18:32.400
So uh I would like to end the presentation uh saying hey when we are developing harnesses when it comes to group agents we need to keep in mind three additional things at least.

s139
00:18:32.400 --> 00:18:40.960
One of them being the security layer, the gate which keeps information out or filters out the information depending upon the context.

s140
00:18:40.960 --> 00:18:48.720
How we design the memory, the harness and how we route the information to the responsible parties.

s141
00:18:48.720 --> 00:18:50.880
Thank you.

s142
00:19:06.263 --> 00:19:08.263
[music]