WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=cSz7aL2nl2U

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/cSz7aL2nl2U.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:12.440 --> 00:00:13.360
Right.

s2
00:00:13.360 --> 00:00:15.320
Thank you all for for coming.

s3
00:00:15.320 --> 00:00:16.960
Um my name is Gil Feyg.

s4
00:00:16.960 --> 00:00:20.320
I am the co-founder and CTO of Merge.

s5
00:00:20.320 --> 00:00:24.080
Merge helps companies build production AI very easily.

s6
00:00:24.080 --> 00:00:28.040
We are the connective infrastructure uh to make everything come together.

s7
00:00:28.040 --> 00:00:32.240
Uh today I'm going to be talking about why you need a context graph and how to build it.

s8
00:00:32.240 --> 00:00:36.080
What I'm not going to do is talk about specific technology you should be using.

s9
00:00:36.080 --> 00:00:37.800
Uh you can talk to your agent about that.

s10
00:00:37.800 --> 00:00:38.800
It will help you.

s11
00:00:38.800 --> 00:00:41.240
There's a lot of good technologies that can back this.

s12
00:00:41.240 --> 00:00:50.920
Uh the focus of this is uh why you need a context graph and then sort of in a in a higher level how you should structure it, what the components should be, uh and how things come together

s13
00:00:50.920 --> 00:00:56.080
to build the best possible uh brain for your company.

s14
00:00:56.280 --> 00:01:00.040
If you haven't gotten a hat yet, by the way, my team is walking around passing out hats.

s15
00:01:00.040 --> 00:01:02.240
They're They say well connected.

s16
00:01:02.240 --> 00:01:04.839
Um so let's dive in here.

s17
00:01:04.839 --> 00:01:08.640
So I'm not going to spend much talking about Merge, but just a quick overview of us.

s18
00:01:08.640 --> 00:01:20.120
Um we have over 400 enterprise customers, OpenAI, Ramp, Netflix, Perplexity, a lot of companies across the board using us for both internal and customer-facing AI use cases.

s19
00:01:20.120 --> 00:01:25.200
Uh we have three different products, Unified, Agent Handler, and Gateway that cover a lot of the components of it.

s20
00:01:25.200 --> 00:01:28.040
Um we were founded 6 years ago by me and my co-founder Shen Shi.

s21
00:01:28.040 --> 00:01:29.320
We've raised three rounds.

s22
00:01:29.320 --> 00:01:34.960
Uh we're 130 employees in the three most beautiful cities in the world.

s23
00:01:35.560 --> 00:01:36.120
All right.

s24
00:01:36.120 --> 00:01:38.040
So quick agenda here.

s25
00:01:38.040 --> 00:01:44.960
Step one or number one, we will talk about what a context graph even is and why it's so important that every company has one.

s26
00:01:44.960 --> 00:01:53.360
We'll go into the different tiers of context, what what types of data, what types of of uh sort of logic from your company should be making it into your graph.

s27
00:01:53.360 --> 00:01:55.800
We'll talk about context selection strategy.

s28
00:01:55.800 --> 00:02:01.440
How are we picking the most relevant content for our agents or contexts for our agents at any moment in time?

s29
00:02:01.440 --> 00:02:03.040
Traceability and provenance.

s30
00:02:03.040 --> 00:02:08.479
How do we know where that data came from, who requested it, when it when it's from, is it fresh, all of that.

s31
00:02:08.479 --> 00:02:11.520
And then lastly, some final key takeaways.

s32
00:02:11.520 --> 00:02:14.080
At the end, I'll stick around if you do have any questions.

s33
00:02:14.080 --> 00:02:17.760
Uh so, feel free to to come up to me afterwards.

s34
00:02:18.680 --> 00:02:19.040
All right.

s35
00:02:19.040 --> 00:02:21.800
So, what is a context graph in this talk?

s36
00:02:21.800 --> 00:02:28.280
We're not going to go into the nodes and the edges that connect each node in a true graph infrastructure.

s37
00:02:28.280 --> 00:02:31.240
We're going to talk about the company brain.

s38
00:02:31.240 --> 00:02:35.600
Your company context graph consists of a lot of different components.

s39
00:02:35.600 --> 00:02:37.080
It has third-party data.

s40
00:02:37.080 --> 00:02:41.840
What are the other systems you use, like NetSuite, uh like like Jira.

s41
00:02:41.840 --> 00:02:44.200
Um it has structured context.

s42
00:02:44.200 --> 00:02:47.520
Uh so, that could be uh data coming in from other third parties.

s43
00:02:47.520 --> 00:02:52.320
That could be data that that you all have put into your context layer.

s44
00:02:52.320 --> 00:02:57.360
Uh things like things like static documents that you want to always be passed to your agents.

s45
00:02:57.360 --> 00:02:58.240
We have memories.

s46
00:02:58.240 --> 00:02:59.360
These are things that are formed.

s47
00:02:59.360 --> 00:03:06.440
We all know memory, things that are formed as you're using the agent, telling it to remember things, things that your agent thinks that it should remember on its own.

s48
00:03:06.440 --> 00:03:08.400
And then we also have skills.

s49
00:03:08.400 --> 00:03:14.280
Uh and skills are a very very important part of your context graph, which we'll get to.

s50
00:03:15.640 --> 00:03:16.000
All right.

s51
00:03:16.000 --> 00:03:17.560
I see some picture Okay, cool.

s52
00:03:17.560 --> 00:03:18.519
Uh so, moving on.

s53
00:03:18.519 --> 00:03:20.080
Let's get Let's go through an example here.

s54
00:03:20.080 --> 00:03:29.640
We're going to use this example to guide why it's not it's not enough to just connect a single MCP server or a couple MCP servers and call it a day.

s55
00:03:29.640 --> 00:03:31.680
So, let's ask this simple question.

s56
00:03:31.680 --> 00:03:38.440
One of one of your employees or or customers asks your agent, "Why was customer A upset last week?"

s57
00:03:38.440 --> 00:03:40.320
Pretty simple question to answer.

s58
00:03:40.320 --> 00:03:48.320
The agent goes, it sees it has access to your Zendesk ticketing, and it can check and say, "Hey, what it what tickets were filed by this customer last week?"

s59
00:03:48.320 --> 00:03:48.640
Great.

s60
00:03:48.640 --> 00:03:54.640
We saw that they saw that the API sync was broken, um and specific details about what went wrong.

s61
00:03:54.640 --> 00:03:58.080
We also see the customer filed three tickets total, all of that.

s62
00:03:58.080 --> 00:04:00.040
Great, that answers that question.

s63
00:04:00.040 --> 00:04:01.400
But what if we want to extend that?

s64
00:04:01.400 --> 00:04:05.120
And what if we want to say, "Which customers were upset last week?"

s65
00:04:05.120 --> 00:04:12.480
That becomes a lot harder because you now have to have your agent load in every single ticket from Zendesk that happened in the last week.

s66
00:04:12.480 --> 00:04:15.840
What if we're saying, "Which customers were upset in the last year?"

s67
00:04:15.840 --> 00:04:16.959
We should be able to answer that.

s68
00:04:16.959 --> 00:04:26.840
Our agent should be able to answer that, but it can't if it has to live live look up and pull tickets 100 at a time, thousands and thousands of those until it times out and it just can't answer the question.

s69
00:04:26.840 --> 00:04:38.240
So, MCP is not fully solving a lot of these problems because the underlying access patterns of the APIs of these other platforms don't support a live look up methodology.

s70
00:04:38.240 --> 00:04:47.440
If you could ask your ticketing system something semantically, if Jira exposed an endpoint that said, you know, that you could query in English, "Which customers were upset last week?"

s71
00:04:47.440 --> 00:04:49.880
and it returned that data, that would be great.

s72
00:04:49.880 --> 00:04:58.480
But no platforms incentivize to do that because it costs them money to vectorize it and it makes it so you don't have to visit their platform as often.

s73
00:04:59.280 --> 00:05:01.320
So, you need a sinking layer on top of this.

s74
00:05:01.320 --> 00:05:11.360
You can't just use MCP to live look up for things that you want to go deeper into, that you want to semantically question, you need to actually sync a copy of the data locally.

s75
00:05:11.360 --> 00:05:11.800
Okay?

s76
00:05:11.800 --> 00:05:13.800
This doesn't apply to everything.

s77
00:05:13.800 --> 00:05:17.360
So, you might have you know, let's say static look ups to Stripe.

s78
00:05:17.360 --> 00:05:20.480
We're always going to be asking Stripe a question about a specific customer.

s79
00:05:20.480 --> 00:05:22.800
We're always going to be taking a single action in Stripe.

s80
00:05:22.800 --> 00:05:23.600
That can be MCP.

s81
00:05:23.600 --> 00:05:24.560
That can be live.

s82
00:05:24.560 --> 00:05:33.760
But if you need any form of semantic look ups, any form of analysis that is beyond a simple look up, simple question, but requires looking across a a vast data set,

s83
00:05:33.760 --> 00:05:35.880
you have to sync a copy.

s84
00:05:35.880 --> 00:05:40.480
So, that becomes the next part of your sinking layer of your of your, uh, context layer.

s85
00:05:40.480 --> 00:05:46.160
So, you have your MCPs, your live look ups, you have your synced or your cached look ups as well.

s86
00:05:46.160 --> 00:05:50.560
And that you're pulling in from a third-party, you're putting in a vector database locally.

s87
00:05:50.560 --> 00:05:54.080
Again, AI can build that vector DB for you relatively easily.

s88
00:05:54.080 --> 00:06:01.760
Uh you got to you got to you know, work on your strategy of how you store data in the context layer and how you retrieve from it uh for that sync to cache layer.

s89
00:06:01.760 --> 00:06:04.800
Um but that's sort of the next component here.

s90
00:06:04.800 --> 00:06:07.880
But the next problem is, all right, now we have all this relevant data.

s91
00:06:07.880 --> 00:06:14.760
We're we're pulling from our vector database to get, you know, semantic data and we're doing live lookups for other data we might want on top.

s92
00:06:14.760 --> 00:06:21.440
Now, we need to bring all that data together cuz we can't just dump a massive amount of retrieved context back to our agent.

s93
00:06:21.440 --> 00:06:23.760
So, that's where we introduce the router and the summarizer.

s94
00:06:23.760 --> 00:06:26.440
The router is sending things to the right place at the beginning.

s95
00:06:26.440 --> 00:06:27.520
The prompt comes in.

s96
00:06:27.520 --> 00:06:31.080
It decides where, you know, who should be handling this, where it should be sent.

s97
00:06:31.080 --> 00:06:44.840
Once that data's all aggregated in there, our summarizer then takes that data from from the the synced context and from the live context, summarizes it, and sends it back to the agent that sent the prompt over.

s98
00:06:44.919 --> 00:06:49.520
But you all notice, we have one thing here that we haven't talked about, and that's memory and skills.

s99
00:06:49.520 --> 00:06:50.480
That could live separately.

s100
00:06:50.480 --> 00:06:52.800
Why is it important for your context layer?

s101
00:06:52.800 --> 00:07:00.400
The reason is that getting context is actually a set of company principles and processes in and of itself.

s102
00:07:00.400 --> 00:07:06.120
For example, which customer was upset last week can mean something very different to one company versus another.

s103
00:07:06.120 --> 00:07:08.320
It could be number of tickets that are opened.

s104
00:07:08.320 --> 00:07:10.360
Um but actually, what if those tickets were positive?

s105
00:07:10.360 --> 00:07:19.680
What if actually we don't use a ticketing system and that we know that for customer happiness, we need to actually always go look at Qualtrics first and see survey data cuz that's just how our company works.

s106
00:07:19.680 --> 00:07:20.840
We're different.

s107
00:07:20.840 --> 00:07:25.520
That's why skills are important because you need skills like how do we determine if a customer is happy?

s108
00:07:25.520 --> 00:07:36.600
So, now your agent should first check the skills before it then goes and starts checking your synced layer, your your uh live layer for what data to get and then summarizing and returning it.

s109
00:07:36.600 --> 00:07:40.720
So, the flow here is prompt comes in, gets routed to the right agent.

s110
00:07:40.720 --> 00:07:47.160
The agent looks to see if it has any memory or skills associated with the question or the task at hand.

s111
00:07:47.160 --> 00:07:56.120
Based on that, it then knows where to look in the synced cache DB and from the live MCP lookups, summarize, send back.

s112
00:07:57.200 --> 00:07:58.160
There's no magic here.

s113
00:07:58.160 --> 00:07:59.160
There really is none.

s114
00:07:59.160 --> 00:08:05.880
It's all physics as long as you know how to calculate, you know, within within your own sort of use case, you're great.

s115
00:08:05.880 --> 00:08:15.600
This flow here where we looked at the number of open tickets again would not have been possible with live lookups, but with a local lookup it was really simple and we were able to basically gather all customers,

s116
00:08:15.600 --> 00:08:20.560
number of tickets they have, summarize their tickets, and then give a status on each one.

s117
00:08:20.560 --> 00:08:23.520
So, it's a mix of some live lookups, some cache lookups.

s118
00:08:23.520 --> 00:08:29.919
That's all coming together here to build really really rich data for any question that gets asked.

s119
00:08:31.800 --> 00:08:32.640
All right.

s120
00:08:32.640 --> 00:08:38.640
And so, there are different tiers of context and some might debate whether prompt skills and memory are context.

s121
00:08:38.640 --> 00:08:41.039
Context is anything that's being passed into the agent.

s122
00:08:41.039 --> 00:08:45.000
We're going to put this all here because they all belong in your context layer.

s123
00:08:45.000 --> 00:08:46.400
So, you have your prompt skills and memory.

s124
00:08:46.400 --> 00:08:50.880
We talked about how those are going to really drive the the where do we get data from?

s125
00:08:50.880 --> 00:08:52.640
You have your live API calls.

s126
00:08:52.640 --> 00:08:57.600
These are easy easier to build, easier to add on, they're cheaper to run than synced context.

s127
00:08:57.600 --> 00:09:02.960
So, you should prefer these when you know you're going to have simple lookups, simple retrievals.

s128
00:09:02.960 --> 00:09:12.839
Cached data is absolutely essential if you want that that sort of semantic data, the complex lookups, deep deep questions about about any data.

s129
00:09:12.839 --> 00:09:16.760
But, keep in mind it's a little bit more complicated to set up and build.

s130
00:09:16.760 --> 00:09:21.480
It's more expensive to run because you have to have a vector database and you're constantly having to sync that data.

s131
00:09:21.480 --> 00:09:24.200
So, it is tougher to keep up to date.

s132
00:09:24.200 --> 00:09:33.920
And then lastly, something we didn't quite discuss yet is derived context, which is basically any additional context you create when you're pulling in data from the third party.

s133
00:09:33.920 --> 00:09:43.480
So, this could be, you You as we're live pulling in all tickets, we're also for every time a customer's full set of tickets comes in, we're summarizing those and we're actually storing

s134
00:09:43.480 --> 00:09:50.200
in our stored context, our structured stored context, we're storing a summary of the data that we pulled in.

s135
00:09:50.200 --> 00:09:57.600
So, that in the future, if we have a common question, we don't have to then also go go re-look up and re-summarize everything.

s136
00:09:58.480 --> 00:09:58.960
Awesome.

s137
00:09:58.960 --> 00:10:00.480
So, all four of those come together.

s138
00:10:00.480 --> 00:10:02.960
You saw the the chart for how they do that.

s139
00:10:02.960 --> 00:10:04.840
Um and yeah, those are the tiers.

s140
00:10:04.840 --> 00:10:09.800
So, quick flowchart of how this works when when a query comes in or a prompt comes in.

s141
00:10:09.800 --> 00:10:13.080
So, first thing here is do we have a skill or memory match?

s142
00:10:13.080 --> 00:10:16.240
If so, yes, we want to execute that skill.

s143
00:10:16.240 --> 00:10:18.000
Does the skill need external data?

s144
00:10:18.000 --> 00:10:22.600
No, then we're just generating a response and we're sending that right back.

s145
00:10:22.600 --> 00:10:24.680
Unfortunately, it's not always this easy.

s146
00:10:24.680 --> 00:10:27.960
So, what happens if we don't have a skill or memory match?

s147
00:10:27.960 --> 00:10:28.360
No?

s148
00:10:28.360 --> 00:10:31.160
Well, then let's go right to to data selection here.

s149
00:10:31.160 --> 00:10:33.760
Uh and same thing for if that skill needs external data.

s150
00:10:33.760 --> 00:10:40.120
So, we say yes, the first thing we're doing is checking our cache or our synced data store and we're saying, is the data here fresh?

s151
00:10:40.120 --> 00:10:41.320
Is it within our SLAs?

s152
00:10:41.320 --> 00:10:44.280
Can we use this data cuz it's viewed as fresh enough?

s153
00:10:44.280 --> 00:10:47.400
Yes, then let's use that cache context.

s154
00:10:47.400 --> 00:10:48.200
No?

s155
00:10:48.200 --> 00:10:52.920
Then, unfortunately, we have to go down and we have to call the live API.

s156
00:10:52.920 --> 00:10:58.480
Now, once we do have enough context, we're going to loop through these a few times till we build up the context we need.

s157
00:10:58.480 --> 00:11:09.600
Once we do have enough context, we might want to store a memory of of whatever we just we just discovered or or figured out, uh store that to memory and then generate the response and send it back.

s158
00:11:09.600 --> 00:11:19.520
And then, same thing here, if we didn't have enough context, I I just showed that, but we have to call the live API and then we will we will uh generate that response back to the client.

s159
00:11:21.520 --> 00:11:22.520
I'll leave it here for a second.

s160
00:11:22.520 --> 00:11:25.960
I I see some people looking and taking pictures.

s161
00:11:30.240 --> 00:11:31.200
All right.

s162
00:11:31.200 --> 00:11:35.920
So, last piece here, pretty much at the end, traceability and provenance.

s163
00:11:35.920 --> 00:11:38.480
Doesn't seem important at front up front.

s164
00:11:38.480 --> 00:11:41.160
Do not drop where the data is coming from.

s165
00:11:41.160 --> 00:11:42.640
Always store it.

s166
00:11:42.640 --> 00:11:44.240
You you have that from MCP.

s167
00:11:44.240 --> 00:11:48.760
You also have that in if for anything you're syncing in Vector DBs, there's something called metadata.

s168
00:11:48.760 --> 00:11:49.960
You want to store it in there.

s169
00:11:49.960 --> 00:11:51.600
It's associated with the data.

s170
00:11:51.600 --> 00:11:52.320
It is critical.

s171
00:11:52.320 --> 00:12:02.080
It's similar to, you know, the way that this surface is in ChatGPT or any surface that you're using is when it tells you a fact or it says something, at the end it says three sources and it links to where it came from.

s172
00:12:02.080 --> 00:12:06.080
That is even more important when you're making business critical decisions off of that data.

s173
00:12:06.080 --> 00:12:12.440
When that data is is versioned, when that data can become outdated, when you're letting it make a decision on data that it synced a year ago.

s174
00:12:12.440 --> 00:12:13.680
You don't want that.

s175
00:12:13.680 --> 00:12:15.240
So, you need to know where it came from.

s176
00:12:15.240 --> 00:12:16.200
Where did we fetch it?

s177
00:12:16.200 --> 00:12:17.040
When did we fetch it?

s178
00:12:17.040 --> 00:12:17.800
Which is not here.

s179
00:12:17.800 --> 00:12:21.080
Under what idea under what ID and what scopes?

s180
00:12:21.080 --> 00:12:22.400
Who accessed the data?

s181
00:12:22.400 --> 00:12:24.440
What permissions were used to access it?

s182
00:12:24.440 --> 00:12:25.400
Was it transformed?

s183
00:12:25.400 --> 00:12:27.240
What did it originally look like?

s184
00:12:27.240 --> 00:12:28.840
And then lastly, what should invalidate it?

s185
00:12:28.840 --> 00:12:30.680
That could be age, for example.

s186
00:12:30.680 --> 00:12:35.600
The data is more than more than a month old, we don't want to consider this ever.

s187
00:12:35.600 --> 00:12:44.520
Really important for showing in your platform and if not for showing in your internal platform or whatever you're building, even for for being able to investigate when something does go wrong,

s188
00:12:44.520 --> 00:12:46.800
cuz it will.

s189
00:12:47.600 --> 00:12:51.680
So, putting it all together, here's another example of one that we might use.

s190
00:12:51.680 --> 00:12:55.040
So, give me a one paragraph status update on our customer Acme.

s191
00:12:55.040 --> 00:12:58.240
Include renewal risk and last support interaction.

s192
00:12:58.240 --> 00:13:00.839
First, we're going to check the cache for the account summary.

s193
00:13:00.839 --> 00:13:02.680
We're going to assume there's no skills here.

s194
00:13:02.680 --> 00:13:04.080
So, we'll check that.

s195
00:13:04.080 --> 00:13:09.480
If it's still or missing, we're going to call the CRM using a live API lookup.

s196
00:13:09.480 --> 00:13:12.760
Then, we'll query the ticketing system for the last support interaction.

s197
00:13:12.760 --> 00:13:16.800
Again, that's cached if within the SLA, otherwise it might have to do a live lookup.

s198
00:13:16.800 --> 00:13:18.400
We're only we only want to pull five tickets.

s199
00:13:18.400 --> 00:13:23.680
We can't pull the full history because again, when we have live lookups, we run into timeout risk.

s200
00:13:23.680 --> 00:13:26.000
And then lastly, we'll generate a derived summary.

s201
00:13:26.000 --> 00:13:30.400
We might store that derived summary, and we'll pass that back.

s202
00:13:33.200 --> 00:13:36.240
So, key takeaways, tools are not context.

s203
00:13:36.240 --> 00:13:39.360
A context graph is going to help you reason across the whole company.

s204
00:13:39.360 --> 00:13:40.320
It's not just tools.

s205
00:13:40.320 --> 00:13:40.920
It's data.

s206
00:13:40.920 --> 00:13:42.040
It's synced data.

s207
00:13:42.040 --> 00:13:44.520
It's processes that you have as a business.

s208
00:13:44.520 --> 00:13:48.280
Who is documenting, you know, when a customer complains, what do we do?

s209
00:13:48.280 --> 00:13:50.960
How does it know to always route to the right place?

s210
00:13:50.960 --> 00:13:52.000
That is a skill.

s211
00:13:52.000 --> 00:13:53.680
These It's your company's skills.

s212
00:13:53.680 --> 00:13:54.760
It's your processes.

s213
00:13:54.760 --> 00:13:58.120
It's your data, and it's your tools and actions you need to take.

s214
00:13:58.120 --> 00:13:59.880
More data, not always better.

s215
00:13:59.880 --> 00:14:01.840
You want fresh, high-quality data.

s216
00:14:01.840 --> 00:14:03.560
You want to pull only what you need.

s217
00:14:03.560 --> 00:14:08.360
When you do have to sync data, you want to structure it properly, so it's accessed in the right way.

s218
00:14:08.360 --> 00:14:11.440
And then lastly, every answer has to be traceable.

s219
00:14:11.440 --> 00:14:18.280
You do not want to be left in a position where your AI made a bad decision, and you have no idea why it did that.

s220
00:14:19.280 --> 00:14:22.040
So, last piece, plugging merch here.

s221
00:14:22.040 --> 00:14:23.360
Enterprise AI runs on us.

s222
00:14:23.360 --> 00:14:25.720
We power these features for a lot of companies.

s223
00:14:25.720 --> 00:14:27.800
We have Gateway, which is an LLM router.

s224
00:14:27.800 --> 00:14:32.960
We have Unified, which helps you sync data from third parties out of your customers' platforms.

s225
00:14:32.960 --> 00:14:46.760
And then we have Agent Handler, which is live MCP connections pre-built across hundreds of tools, governed identity all built in, uh including provenance, uh and yeah, it's plug and play for both internal and customer-facing use cases.

s226
00:14:46.760 --> 00:14:51.320
So, if you want to build an easy context layer, hit us up.

s227
00:14:51.320 --> 00:14:53.960
Thank you, everyone, for listening.

s228
00:15:09.150 --> 00:15:11.150
[music]