WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=u6jJcIFDLE4

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/u6jJcIFDLE4.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.560 --> 00:00:14.240
Uh okay, so I'm Subbiah.

s3
00:00:14.240 --> 00:00:16.840
I had AI engineering at ZS.

s4
00:00:16.840 --> 00:00:17.760
Uh I'm Abhilash.

s5
00:00:17.760 --> 00:00:20.000
I'm director of AI

s6
00:00:20.000 --> 00:00:21.400
So, ZS, we are a tech firm.

s7
00:00:21.400 --> 00:00:25.720
We work with many of the top companies in the world including a lot of the top pharmas actually.

s8
00:00:25.720 --> 00:00:29.640
Um so, today's talk I think we wanted to as I think already introduced, right?

s9
00:00:29.640 --> 00:00:31.600
We wanted to talk from our experience, right?

s10
00:00:31.600 --> 00:00:33.600
Building multi-agent pipelines.

s11
00:00:33.600 --> 00:00:35.480
Uh what are the mistakes we did, right?

s12
00:00:35.480 --> 00:00:38.400
And what did we learn and how did we fix them?

s13
00:00:38.400 --> 00:00:46.640
So, um so, I'm going to orient it more on pharma commercial domain and for people in the room probably who are not aware quickly um pharma has two main functions.

s14
00:00:46.640 --> 00:00:47.800
One is R&D, right?

s15
00:00:47.800 --> 00:00:49.160
The drug discovery, right?

s16
00:00:49.160 --> 00:00:50.520
And the clinical trials part.

s17
00:00:50.520 --> 00:00:51.480
And then there's a commercial.

s18
00:00:51.480 --> 00:00:54.640
How do you take a drug to a patient basically?

s19
00:00:54.640 --> 00:00:58.000
And and within commercial there are different functions.

s20
00:00:58.000 --> 00:00:59.960
Like once you create a drug, right?

s21
00:00:59.960 --> 00:01:01.880
Um what is the performance of a brand?

s22
00:01:01.880 --> 00:01:04.839
How is the drug performing in different markets, right?

s23
00:01:04.839 --> 00:01:07.240
Uh then there are things around your field force, your reps.

s24
00:01:07.240 --> 00:01:09.360
How effectively are they engaging uh right?

s25
00:01:09.360 --> 00:01:10.440
With everyone?

s26
00:01:10.440 --> 00:01:11.840
There are things around patient journey.

s27
00:01:11.840 --> 00:01:14.080
How a patient are adopting a drug, right?

s28
00:01:14.080 --> 00:01:16.920
I think if there's any therapy switch which is happening.

s29
00:01:16.920 --> 00:01:21.920
So, as you can think about there is a lot of analytics which really happens in a commercial domain.

s30
00:01:21.920 --> 00:01:25.680
And how do typically analysts work?

s31
00:01:25.680 --> 00:01:26.880
So, there are four steps, right?

s32
00:01:26.880 --> 00:01:28.080
What analysts do, right?

s33
00:01:28.080 --> 00:01:31.880
So, first there is always something called signal detection, right?

s34
00:01:31.880 --> 00:01:35.720
So, signal can be in something like okay, the prescriptions what a doctor is writing.

s35
00:01:35.720 --> 00:01:37.880
Maybe is there is there a drop in the prescription?

s36
00:01:37.880 --> 00:01:39.800
So, that's a signal.

s37
00:01:39.800 --> 00:01:44.920
So, once you got a signal, the second thing what an analyst does is why is the signal really failing?

s38
00:01:44.920 --> 00:01:45.800
What is the reason for it?

s39
00:01:45.800 --> 00:01:48.240
So, is it like there is a competitor drug which has come in?

s40
00:01:48.240 --> 00:01:50.480
Because of that is it is it reducing?

s41
00:01:50.480 --> 00:01:53.840
Is it is it because maybe a payer coverage for the drug has reduced?

s42
00:01:53.840 --> 00:02:00.920
Or maybe the the reps on the ground does not proper uh they are actually not taking the benefits to the doctors.

s43
00:02:00.960 --> 00:02:05.360
And once you arrive at the reason, the next step becomes, "Okay, what is the action do you take?"

s44
00:02:05.360 --> 00:02:12.400
So, if reps Suppose if reps the coverage is not good in a particular region, should do we have to increase that?

s45
00:02:12.400 --> 00:02:15.000
And once you do that, what is the what is the outlook, right?

s46
00:02:15.000 --> 00:02:18.360
Is my brand is my sales performance is is it going to improve, right?

s47
00:02:18.360 --> 00:02:21.400
So, these are the four things which happens.

s48
00:02:21.560 --> 00:02:26.440
Now, for some of the top farmers, what we have done is how do we in an agentic way, right?

s49
00:02:26.440 --> 00:02:29.600
I think how do we actually mimic this this analytics work?

s50
00:02:29.600 --> 00:02:33.000
So, what we did, we built agents for every step, right?

s51
00:02:33.000 --> 00:02:36.160
Signal detection, we said, "Okay, we'll have an agent for signal detection.

s52
00:02:36.160 --> 00:02:38.959
It'll identify identify the signals for me."

s53
00:02:38.959 --> 00:02:40.720
Second, what is the root cause, right?

s54
00:02:40.720 --> 00:02:42.200
For the signals, right?

s55
00:02:42.200 --> 00:02:43.720
So, in this case, we have two agents.

s56
00:02:43.720 --> 00:02:45.520
One, we call it a source localization.

s57
00:02:45.520 --> 00:02:53.560
So, for example, if my sales is dropping at a national level, is it because it's dropping at say a particular region, or is it dropping for a payer?

s58
00:02:53.560 --> 00:02:54.640
So, we need to understand that.

s59
00:02:54.640 --> 00:02:57.120
We are We are trying to identify the source of it.

s60
00:02:57.120 --> 00:02:59.760
And then, once we understand what is the real reason, right?

s61
00:02:59.760 --> 00:03:03.080
I think the the sales performance has gone down.

s62
00:03:03.080 --> 00:03:05.600
That's another agent, the driver attribution agent.

s63
00:03:05.600 --> 00:03:07.680
And then, the last step is your synthesis, right?

s64
00:03:07.680 --> 00:03:13.560
So, depending on the cause, now what is the action you have to take and what is the outlook, like typically how an analyst used to do.

s65
00:03:13.560 --> 00:03:18.640
And all of this, we used to have an orchestrator agent which connects all these agents together.

s66
00:03:18.640 --> 00:03:22.160
So, now, what happened once we had the system, what did it generate, right?

s67
00:03:22.160 --> 00:03:25.600
It generated something an information packet like this, right?

s68
00:03:25.600 --> 00:03:27.320
It clearly tells you the signal, right?

s69
00:03:27.320 --> 00:03:32.200
So, first it says, "Maybe my brand's prescriptions have dropped 18% in some territory, right?

s70
00:03:32.200 --> 00:03:34.720
In some timeframe, maybe 4 weeks."

s71
00:03:34.720 --> 00:03:36.000
It then tells you the reason.

s72
00:03:36.000 --> 00:03:38.200
Why did it Why did it drop, right?

s73
00:03:38.200 --> 00:03:41.959
The reason it says because a payer actually, right?

s74
00:03:41.959 --> 00:03:45.920
The coverage for this drug has actually they moved it to a lower tier.

s75
00:03:45.920 --> 00:03:49.880
So, for patient it's expensive actually to to buy this drug.

s76
00:03:49.880 --> 00:03:57.200
The action it says, "Okay, because doctors are writing less prescriptions, maybe send more sales reps to talk to doctors, right?

s77
00:03:57.200 --> 00:04:00.040
And increase the number of prescriptions you're writing.

s78
00:04:00.040 --> 00:04:04.960
And then if you take this action, maybe your outlook your sales performance is going to increase.

s79
00:04:04.960 --> 00:04:11.720
So, all of this looks good high level, but then if you if you look at at it closely, it's not very coherent, right?

s80
00:04:11.720 --> 00:04:12.640
The cause is right.

s81
00:04:12.640 --> 00:04:14.280
It identified the right cause, right?

s82
00:04:14.280 --> 00:04:17.120
Because patients can't afford the drug drug, right?

s83
00:04:17.120 --> 00:04:22.280
But the action it said it didn't really focus on the payer part, the insurance part of it, right?

s84
00:04:22.280 --> 00:04:24.080
It just said reps, right?

s85
00:04:24.080 --> 00:04:26.560
Uh focus more reps actually, right?

s86
00:04:26.560 --> 00:04:30.080
And then the outlook because the action is wrong, the outlook is not going to match.

s87
00:04:30.080 --> 00:04:39.960
So, at each level if you see, it has actually derived the right fact, but then there is no single agent which is owning which understands the end-to-end picture basically.

s88
00:04:40.280 --> 00:04:41.800
So, why did this happen?

s89
00:04:41.800 --> 00:04:44.080
Like why did it fail, right?

s90
00:04:44.080 --> 00:04:46.760
So, obviously it's not the LLM which failed, right?

s91
00:04:46.760 --> 00:04:48.880
It's the way how we split the work, right?

s92
00:04:48.880 --> 00:04:52.880
Because we tried mimicking the analyst behavior, and we did it.

s93
00:04:52.880 --> 00:04:58.800
The first first key issue is like a language model is actually determining your signals.

s94
00:04:58.800 --> 00:05:05.480
So, signals like things like your sales drop is a simple information which you can use statistical methods to actually go and fetch this information.

s95
00:05:05.480 --> 00:05:07.360
You don't need a language model actually, right?

s96
00:05:07.360 --> 00:05:09.520
To fetch this information.

s97
00:05:09.520 --> 00:05:17.120
Second is as your multi agents, there is a lot of context hand off which is happening, and context is actually getting lost at each of these hand offs.

s98
00:05:17.120 --> 00:05:26.919
So, for example, the driver attribution agent is actually determining the right cause, but then the next agent, the synthesis agent, is actually is not able to understand why is that, right?

s99
00:05:26.919 --> 00:05:29.480
The the payers are finding the drug to be expensive.

s100
00:05:29.480 --> 00:05:33.960
It's not understanding the weightage of an insurance coverage going going down.

s101
00:05:33.960 --> 00:05:35.480
So, that is a key information, right?

s102
00:05:35.480 --> 00:05:37.000
Which is getting lost.

s103
00:05:37.000 --> 00:05:42.800
And the last big piece is there is no shared understanding of the business domain knowledge for all these agents.

s104
00:05:42.800 --> 00:05:45.160
All these agents don't understand metrics, right?

s105
00:05:45.160 --> 00:05:50.000
So, things like TRX, the number of transaction the number of prescriptions which a doctor writes, right?

s106
00:05:50.000 --> 00:05:51.600
What is the relationship between them?

s107
00:05:51.600 --> 00:05:53.760
Why does it go up or down?

s108
00:05:53.760 --> 00:05:55.320
So, those are the the reasons.

s109
00:05:55.320 --> 00:05:57.120
So, then what did we do?

s110
00:05:57.120 --> 00:06:01.200
Uh so, I called Abhilash to come and solve for this.

s111
00:06:01.200 --> 00:06:03.440
Uh thank you, Sufiyan.

s112
00:06:03.440 --> 00:06:05.320
So, we had like three problems.

s113
00:06:05.320 --> 00:06:08.040
Uh so, we are thinking how are we going to solve this?

s114
00:06:08.040 --> 00:06:12.120
Uh our first instinct was we'll go back to the drawing board.

s115
00:06:12.120 --> 00:06:13.720
Uh we'll start designing it again.

s116
00:06:13.720 --> 00:06:15.360
So, maybe the topology was wrong.

s117
00:06:15.360 --> 00:06:16.360
The skills were wrong.

s118
00:06:16.360 --> 00:06:18.880
The tools were wrong.

s119
00:06:18.880 --> 00:06:22.560
Or maybe the maybe we have to dis- define a better handoff.

s120
00:06:22.560 --> 00:06:26.800
Design a better handoff for a better schema between agents.

s121
00:06:26.919 --> 00:06:27.880
But we took a step back.

s122
00:06:27.880 --> 00:06:29.760
We didn't do any of that.

s123
00:06:29.760 --> 00:06:33.800
We like all of us, we went back to Cloud Code.

s124
00:06:33.800 --> 00:06:38.400
So, we opened a very plain empty directory.

s125
00:06:38.400 --> 00:06:39.880
I ran Cloud Code.

s126
00:06:39.880 --> 00:06:43.200
Then give it just bash and the database.

s127
00:06:43.200 --> 00:06:46.360
Then give it an actual signal which we identified.

s128
00:06:46.360 --> 00:06:49.240
Then started observing what it is doing.

s129
00:06:49.240 --> 00:06:55.160
So, while we look at what code does, we are able to figure out fix for all the three issues we discussed.

s130
00:06:55.240 --> 00:06:58.280
The first the first part.

s131
00:06:58.480 --> 00:07:05.200
So, what was happening is the agent was looking at data and deciding on a signal.

s132
00:07:05.200 --> 00:07:07.480
Sometimes it applied some statistical methods.

s133
00:07:07.480 --> 00:07:10.320
Sometimes it barely look at the data and say this is a signal.

s134
00:07:10.320 --> 00:07:11.960
Sometimes it's actually a signal.

s135
00:07:11.960 --> 00:07:14.000
Sometimes it's a noise.

s136
00:07:14.000 --> 00:07:16.120
This is something we don't want an agent to do.

s137
00:07:16.120 --> 00:07:18.520
This is a completely deterministic workflow.

s138
00:07:18.520 --> 00:07:22.000
So, we separated it out from the agentic system.

s139
00:07:22.000 --> 00:07:26.040
So, we built a pure deterministic workflow with different statistical methods.

s140
00:07:26.040 --> 00:07:27.200
We put guardrails.

s141
00:07:27.200 --> 00:07:28.600
We put thresholds.

s142
00:07:28.600 --> 00:07:30.560
We put prioritization.

s143
00:07:30.560 --> 00:07:34.680
Everything happened before the agent even kick starts.

s144
00:07:34.680 --> 00:07:38.960
So, we ran an automated pipeline which scans through the data.

s145
00:07:38.960 --> 00:07:40.960
Identify signals for each of the KPI.

s146
00:07:40.960 --> 00:07:42.200
Is anything happening with that?

s147
00:07:42.200 --> 00:07:43.520
Any anomalies which is happening?

s148
00:07:43.520 --> 00:07:47.640
Any trend which is very uh any trend which is breaking.

s149
00:07:47.640 --> 00:07:50.760
Based on that, we identified a signal, we put it on a queue.

s150
00:07:50.760 --> 00:07:53.680
The moment a signal comes to the queue, the agent wakes up.

s151
00:07:53.680 --> 00:07:58.240
So, the agent's job is to investigate, not to identify.

s152
00:07:59.520 --> 00:08:01.720
The second part.

s153
00:08:01.720 --> 00:08:10.240
So, mainly the issue which we previously what Subair was mentioning, there's no coherence in the output which the agents produced.

s154
00:08:10.240 --> 00:08:11.680
So, we started consolidating.

s155
00:08:11.680 --> 00:08:13.680
We look at how cloud code operates.

s156
00:08:13.680 --> 00:08:18.080
It's able to do a lot of operations, so we started designing around that.

s157
00:08:18.080 --> 00:08:21.640
So, it's repeatedly writing a function and querying database.

s158
00:08:21.640 --> 00:08:24.600
So, we give it a tool for that.

s159
00:08:25.080 --> 00:08:28.720
So, this consolidated the entire process into a single agent.

s160
00:08:28.720 --> 00:08:31.520
That doesn't mean that we didn't do parallelism.

s161
00:08:31.520 --> 00:08:33.120
We still do parallelism.

s162
00:08:33.120 --> 00:08:36.800
What we removed is, do we need distributed reasoning?

s163
00:08:36.800 --> 00:08:39.960
We didn't but the judgment to be distributed between agents.

s164
00:08:39.960 --> 00:08:42.680
That we wanted to consolidate to a single agent.

s165
00:08:42.680 --> 00:08:44.680
So, that what exactly what we did.

s166
00:08:44.680 --> 00:08:52.120
Then occasionally we observed cloud code, it's launching sub-agents dynamically for a very particular focused task.

s167
00:08:52.120 --> 00:08:53.600
So, we did the same thing.

s168
00:08:53.600 --> 00:08:59.800
Because if suppose you want to understand if rapid activity in a particular region, that's an investigation which you need to run.

s169
00:08:59.800 --> 00:09:02.240
That you can still delegate to a sub-agent.

s170
00:09:02.240 --> 00:09:06.200
You can get back the uh results back, not the reasoning or the judgment.

s171
00:09:06.200 --> 00:09:13.360
That is still controlled by the by the main agent, but the investigation and a part of it we did we're delegating it to a sub-agent.

s172
00:09:13.360 --> 00:09:18.360
So, these are some of the things we kept based on observing what cloud was doing cloud code was doing.

s173
00:09:18.360 --> 00:09:23.360
So, that give us a more lighter architecture than what was initially there.

s174
00:09:23.360 --> 00:09:24.880
But still it doesn't solve the problem.

s175
00:09:24.880 --> 00:09:28.080
That's why you see a knowledge graph in the diagram.

s176
00:09:28.080 --> 00:09:31.720
So, it still doesn't have the business context.

s177
00:09:31.720 --> 00:09:41.720
It still didn't doesn't understand all the entities, the domain, the KPIs, how do they relate to each other?

s178
00:09:42.320 --> 00:09:49.360
So, that's something we wanted to solve for because the agent was looking at data, looking at tables, then trying to infer the relationship.

s179
00:09:49.360 --> 00:09:50.960
That which was not scalable.

s180
00:09:50.960 --> 00:09:57.240
And it often produce relationship which which is not actually exist in the data.

s181
00:09:57.800 --> 00:09:59.880
So, what we did.

s182
00:09:59.880 --> 00:10:04.680
So, we've been in this working in this field for a lot lot whole whole lot of years.

s183
00:10:04.680 --> 00:10:06.720
So, we've been doing this for our clients.

s184
00:10:06.720 --> 00:10:09.600
So, we had a lot of domain experts who understand the former domain very well.

s185
00:10:09.600 --> 00:10:11.880
We're doing commercial analytics for the clients.

s186
00:10:11.880 --> 00:10:14.200
So, we start start with them, start building a knowledge graph.

s187
00:10:14.200 --> 00:10:18.160
We start start trying to map out the domain.

s188
00:10:19.520 --> 00:10:21.200
So, we are able to build a knowledge graph.

s189
00:10:21.200 --> 00:10:25.200
We are able to identify different entities, their relationship between them.

s190
00:10:25.200 --> 00:10:27.960
So, if you see, there are like geographic entities.

s191
00:10:27.960 --> 00:10:29.160
There are payers.

s192
00:10:29.160 --> 00:10:33.080
How How geographic entities connect to a payer or an account?

s193
00:10:33.080 --> 00:10:35.040
How does this connect to a brand?

s194
00:10:35.040 --> 00:10:38.520
Then from brand or payer, how does it go to a KPI?

s195
00:10:38.520 --> 00:10:43.360
How does a KPI relate to like a uh secondary KPI, tertiary KPI?

s196
00:10:43.360 --> 00:10:45.600
How does one KPI drives another KPI?

s197
00:10:45.600 --> 00:10:51.000
So, we started mapping out all of this information and then creating our our knowledge graph.

s198
00:10:51.000 --> 00:10:56.680
Once you have the knowledge graph, then we wanted let the agent to navigate the knowledge graph.

s199
00:10:56.680 --> 00:11:01.680
So, the knowledge graph is not just something the agent looks up for data.

s200
00:11:01.680 --> 00:11:04.800
It is a control plane for the agent.

s201
00:11:04.800 --> 00:11:06.520
So, what do you mean by that?

s202
00:11:06.520 --> 00:11:17.320
So, the knowledge graphs dictates what the agent can look into, what path it can take, what investigation hypothesis uh it can evaluate.

s203
00:11:17.320 --> 00:11:24.680
So, for example, if you when you do this analysis, the root of the problem, say something like your TRX is declining at a national level.

s204
00:11:24.680 --> 00:11:26.480
This could be when you do source localization.

s205
00:11:26.480 --> 00:11:31.200
So, that's our uh contact for how to find the where.

s206
00:11:31.200 --> 00:11:32.680
It might happen within a region.

s207
00:11:32.680 --> 00:11:34.720
It could be concentrated in a territory.

s208
00:11:34.720 --> 00:11:38.680
It would be concentrated in a combination of a territory or a pair or an account.

s209
00:11:38.680 --> 00:11:42.240
So, there are whole lot of dimensions which the agent needs to evaluate.

s210
00:11:42.240 --> 00:11:45.280
Now, it's a lot of permission permutation combination.

s211
00:11:45.280 --> 00:11:50.640
So, the knowledge graph guides the agent on how to find where this is concentrated.

s212
00:11:50.640 --> 00:11:53.280
Then the why why part of it.

s213
00:11:53.280 --> 00:11:54.120
Why it is happening?

s214
00:11:54.120 --> 00:11:59.800
When you figure out that okay, something is concentrated at let's say the decline is concentrated at a particular region.

s215
00:11:59.800 --> 00:12:01.720
Now, you need to figure out the why.

s216
00:12:01.720 --> 00:12:05.560
This is where where the KPIs and the relations come comes in.

s217
00:12:05.560 --> 00:12:13.080
Once the agent is able to narrow down the where, then it's able to go and figure out the uh figure out the why part of it.

s218
00:12:13.080 --> 00:12:16.200
Which KPIs is driving what?

s219
00:12:16.200 --> 00:12:18.200
Graph access the control surface.

s220
00:12:18.200 --> 00:12:21.880
So, agent every edge is a hypothesis.

s221
00:12:21.880 --> 00:12:25.520
So, the agent can go and evaluate that hypothe- hypothesis.

s222
00:12:25.520 --> 00:12:27.920
Uh it doesn't go outside of this.

s223
00:12:27.920 --> 00:12:32.880
So, that gives a more bounded surface for the agent to investigate.

s224
00:12:34.560 --> 00:12:38.640
So, like everyone was mentioning about loop, so we also build a loop.

s225
00:12:38.640 --> 00:12:40.040
So, what the agent does?

s226
00:12:40.040 --> 00:12:44.000
So, the agent first start with an entity.

s227
00:12:44.000 --> 00:12:46.320
It goes to the graph.

s228
00:12:46.320 --> 00:12:51.200
It looks at the neighborhood of the Then it it figures out the edges.

s229
00:12:51.200 --> 00:12:53.360
So, it's got some hypothesis.

s230
00:12:53.360 --> 00:12:56.839
It'll go It'll go back to the original data.

s231
00:12:56.839 --> 00:12:59.000
Evaluate that hypothesis.

s232
00:12:59.000 --> 00:13:01.000
Look at the actual numbers.

s233
00:13:01.000 --> 00:13:02.680
Then reason over it.

s234
00:13:02.680 --> 00:13:09.320
Then it will it will either find it find it contradicting or it finding supporting the the evidence supporting the hypothesis.

s235
00:13:09.320 --> 00:13:13.400
If it's supporting it, it started traversing through the graph.

s236
00:13:13.400 --> 00:13:20.720
So, this repeats this until it run out of all the hypothesis or it's able to figure out the root cause.

s237
00:13:20.720 --> 00:13:23.480
So, this this concludes this run.

s238
00:13:23.480 --> 00:13:28.640
This runs and it'll be able to figure out the uh root cause of the problem.

s239
00:13:28.640 --> 00:13:41.400
So, once you build this, so maybe like after like 50 plus turns, a whole lot of tokens, it's able to produce something an analyst was able to produce maybe in three or four weeks in like maybe 20 30 minutes.

s240
00:13:41.400 --> 00:13:43.560
Thank you.

s241
00:13:44.720 --> 00:13:48.480
So, just to wrap it up, key takeaways.

s242
00:13:48.480 --> 00:13:53.120
First thing is I think we should not be introducing human constraints or design constraints into architecture.

s243
00:13:53.120 --> 00:13:56.120
I think let the architecture be derived actually.

s244
00:13:56.120 --> 00:13:57.200
Number one.

s245
00:13:57.200 --> 00:14:01.520
Second, I think any complex workflows will have deterministic parts and agentic parts.

s246
00:14:01.520 --> 00:14:04.680
Don't let agents actually run the deterministic part, right?

s247
00:14:04.680 --> 00:14:08.080
So, I think we need to break break that off.

s248
00:14:08.080 --> 00:14:11.360
The third, need to have one agent, right?

s249
00:14:11.360 --> 00:14:13.680
Which owns the reasoning end to end, right?

s250
00:14:13.680 --> 00:14:19.920
This agent can actually take the call to use sub agents, tools, skills to actually spawn off other other tasks basically, right?

s251
00:14:19.920 --> 00:14:23.040
But you need one agent to own the the reasoning.

s252
00:14:23.040 --> 00:14:28.600
And the last, probably the most important, I think graph cannot be treated just as a lookup layer.

s253
00:14:28.600 --> 00:14:36.200
I think graph has to be treated as a control plane which the agent uses to navigate and takes the next decisions basically.

s254
00:14:36.680 --> 00:14:38.920
Okay, with that, I think thank you.

s255
00:14:38.920 --> 00:14:39.200
Thanks.

s256
00:14:39.200 --> 00:14:40.985
Thanks for attending this.

s257
00:14:40.985 --> 00:14:42.985
[applause]

s258
00:14:57.360 --> 00:14:59.360
[music]