WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=ORVh4GivEU0

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/ORVh4GivEU0.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.800 --> 00:00:13.840
Are we up?

s3
00:00:13.840 --> 00:00:15.599
Okay.

s4
00:00:15.599 --> 00:00:16.320
Can you hear me?

s5
00:00:16.320 --> 00:00:18.480
Just like this just fine, everyone.

s6
00:00:18.480 --> 00:00:18.800
Good.

s7
00:00:18.800 --> 00:00:19.680
Okay.

s8
00:00:19.680 --> 00:00:28.640
Um, so we're we're going to be doing a bit of a speedun here because uh uh we've only got 20 minutes, but um I'm I'm Peter from Unblocked.

s9
00:00:28.640 --> 00:00:32.160
And at Unblocked, we build a context engine.

s10
00:00:32.160 --> 00:00:34.000
So I'm going to show you a couple of things.

s11
00:00:34.000 --> 00:00:36.559
The talk's going to be split into two parts.

s12
00:00:36.559 --> 00:00:41.600
Um I'm going to show you what a context engine is and how it can add value to your organization.

s13
00:00:41.600 --> 00:00:46.800
And then we're going to actually go and build a component of a context engine.

s14
00:00:46.800 --> 00:00:53.520
And I'm going to show you a couple of open source projects that you can uh go and play with after this.

s15
00:00:54.320 --> 00:00:56.719
So there's the agenda.

s16
00:00:56.719 --> 00:00:57.920
What's a context engine?

s17
00:00:57.920 --> 00:00:59.600
How does it deliver value?

s18
00:00:59.600 --> 00:01:01.920
And then we'll build.

s19
00:01:01.920 --> 00:01:08.720
Um so let let's just hop back in time maybe a few years to the times before AI agents.

s20
00:01:08.720 --> 00:01:14.240
And in those days um you were the context layer.

s21
00:01:14.240 --> 00:01:21.759
Imagine that you're a new employee to an organization and you're just trying to get to grips with what's what's happening.

s22
00:01:21.759 --> 00:01:31.680
What you usually end up having to do is look at the look at a whole bunch of code, rip around documentation systems just trying to find the information that you need to get your work done.

s23
00:01:31.680 --> 00:01:43.280
Um, and not to mention like all of the the sort of shared history that's built up over time through an organization dealing with incidents and the battle scars that come with it.

s24
00:01:43.280 --> 00:01:57.680
Um, so what a context engine does is it takes all of that institutional knowledge and it pulls it in and what you get the scattered context that goes into the context engine and out the other end

s25
00:01:57.680 --> 00:02:03.840
you get grounded context which means that the context is all interrelated to each other.

s26
00:02:03.840 --> 00:02:05.840
It's intent specific.

s27
00:02:05.840 --> 00:02:11.200
It's personalized and it's permissions aware.

s28
00:02:12.160 --> 00:02:14.720
And so I'm going to show you exactly what I mean by that.

s29
00:02:14.720 --> 00:02:27.520
Now let's imagine that I'm going to go into here into Claude and I'm going to show you a couple of interesting sessions that I had.

s30
00:02:27.520 --> 00:02:39.440
So um I asked Claude here, this is without a context engine attached to create a plan to optimize a component in our layer called the source mark engine.

s31
00:02:39.440 --> 00:02:44.319
And uh I'm sure you're all familiar with what Claude does when it starts a new task.

s32
00:02:44.319 --> 00:02:45.440
It's like a new employee.

s33
00:02:45.440 --> 00:02:49.519
It doesn't understand anything about your your codebase or your organization.

s34
00:02:49.519 --> 00:02:55.200
So the first thing it does is it reps around your codebase and and tries to figure out what's going on.

s35
00:02:55.200 --> 00:02:57.760
And so you can kind of see that here.

s36
00:02:57.760 --> 00:03:04.720
And you know it figures it out eventually or at least you know it confidently projects that it's figured it out.

s37
00:03:04.720 --> 00:03:09.680
Um, and so it provides kind of like a a recommendation for me there.

s38
00:03:09.680 --> 00:03:14.800
But now I'm going to show you what it looks like when you attach a context engine to the back end.

s39
00:03:14.800 --> 00:03:26.239
So now what it does is it calls the unblock context engine and unblock now delivers a very comprehensive but taskspecific context bundle for this task.

s40
00:03:26.239 --> 00:03:39.440
And what you can see is that it it's referencing not only the source code but also you know notion documents that talk about the architecture um pull requests slack conversations that were had about optimizing

s41
00:03:39.440 --> 00:03:47.840
this particular component um and the the the net out is that you get this really tight context bundle with all these links.

s42
00:03:47.840 --> 00:03:55.840
So now uh if if um you know Claude wants to go back and fetch more information, it has all these great jump off points.

s43
00:03:55.840 --> 00:03:58.879
But the really interesting thing is is this.

s44
00:03:58.879 --> 00:04:08.560
So here I spent um you know a dollar 29 on this and it took about a minute and 30 seconds.

s45
00:04:08.560 --> 00:04:15.680
Um that was with the context engine but without I'm spending up to you know 260 and 3 minutes.

s46
00:04:15.680 --> 00:04:21.519
So what you see essentially is a 50% reduction in cost which is pretty incredible.

s47
00:04:21.519 --> 00:04:32.160
Um now I want to show you another interesting demo um which is uh an interesting session that my colleague Whoops sorry bring this out here.

s48
00:04:32.160 --> 00:04:37.280
It's an interesting um session that my colleague had with unblock the other week.

s49
00:04:37.280 --> 00:04:47.440
So what he we also have a code review product and Richie works on that and he discovered that the number of issues that the code review product was surfacing

s50
00:04:47.440 --> 00:04:49.680
was was starting to drop.

s51
00:04:49.680 --> 00:05:08.000
So he just asked unblocked and unblocked was able to pull this information um and and you know narrow it down very quickly to uh a likely cause which was that we switched from Opus 48 or Opus 46 to Opus 48 and the behavior characteristics

s52
00:05:08.000 --> 00:05:20.160
behind the scenes um you know caused all those issues to drop and uh so what he did is he asked on block to just fix it and now on locked

s53
00:05:20.160 --> 00:05:25.919
um was able to just create a PR behind the scenes and it has all that context and it knows what to do.

s54
00:05:25.919 --> 00:05:37.520
But the really cool part is you can see that it created this description and it's it's just remarkable like it it um understood the PR that caused the regression in the first place

s55
00:05:37.520 --> 00:05:50.880
and then it found a Slack conversation where the conver where where this was being discussed and pulled on some of the data here to help inform um the fix which which is just nuts.

s56
00:05:53.360 --> 00:05:57.919
So, I'm going to go back to wh I think I lost my keynote presentation here.

s57
00:05:57.919 --> 00:05:58.479
Let's see.

s58
00:05:58.479 --> 00:06:00.880
There we go.

s59
00:06:03.840 --> 00:06:07.280
So, you have the same problem with agents, right?

s60
00:06:07.280 --> 00:06:07.840
Cost.

s61
00:06:07.840 --> 00:06:08.400
Oh, sorry.

s62
00:06:08.400 --> 00:06:09.424
This is the wrong

s63
00:06:09.424 --> 00:06:09.840
[laughter]

s64
00:06:09.840 --> 00:06:12.479
It's the wrong keynote.

s65
00:06:17.600 --> 00:06:20.000
There we go.

s66
00:06:20.080 --> 00:06:25.759
So now I think what we're going to do is we're just going to get right into the building part because we're we're quite short on time today.

s67
00:06:25.759 --> 00:06:28.479
Um and so I'm just going to summarize this.

s68
00:06:28.479 --> 00:06:30.560
You've got context scattered all over the place.

s69
00:06:30.560 --> 00:06:37.199
You've got context in PRs, reviews, comments, issues and tickets, docs, conversations.

s70
00:06:37.199 --> 00:06:48.319
Um and and all of that is uh semantically queryable, but it's also structured and rag nails the content part.

s71
00:06:48.319 --> 00:07:00.240
Um, it it can be it can handle things like content questions, find the code, the PR that changed it, the Slack thread that's debating it, and if you wrap that stuff in an agent loop,

s72
00:07:00.240 --> 00:07:02.160
it can go really really far.

s73
00:07:02.160 --> 00:07:15.440
You give an agent loop tools to do file search after this and it it's very powerful but it doesn't handle some of the other type of uh structural questions like PRs that I merged last week

s74
00:07:15.440 --> 00:07:22.479
who reviewed payments the most open PRs etc. Um, and these are queries.

s75
00:07:22.479 --> 00:07:25.520
They're not semantic searches.

s76
00:07:25.840 --> 00:07:39.360
So, I want to just show you again, um, going back to Richie's example here that he started the the conversation with unblock by asking it a query based question because there's a temporal element.

s77
00:07:39.360 --> 00:07:43.840
Can you show me a chart of ratio weekly since January 1st?

s78
00:07:43.840 --> 00:07:44.160
Okay.

s79
00:07:44.160 --> 00:07:51.199
So, in order to do this, it's not going to be able to just pull this information from a vector store because that's not where that information is encoded.

s80
00:07:51.199 --> 00:07:56.960
So, it's it's going to have to do that um through some sort of query engine.

s81
00:07:59.680 --> 00:08:02.479
And so, you know, retrieval has two halves essentially.

s82
00:08:02.479 --> 00:08:07.840
We've got semantic retrieval and that can handle those types of semantic based questions.

s83
00:08:07.840 --> 00:08:11.919
But today, we're going to work on this the structured part.

s84
00:08:11.919 --> 00:08:18.639
We're going to turn structured queries or natural language queries into structured queries.

s85
00:08:20.319 --> 00:08:22.720
But generating the queries is actually the easy part.

s86
00:08:22.720 --> 00:08:25.840
The hard part is the scaffolding that sits around it.

s87
00:08:25.840 --> 00:08:33.760
So you need to do things like understand the schema so you can pass it to the LLM so it generates the a tightly bound query.

s88
00:08:33.760 --> 00:08:36.640
Uh you need to also be able to map identities.

s89
00:08:36.640 --> 00:08:46.800
If I ask a question like open PR is by Rashene, um it's not going to know who Rashene is because his identity in GitHub might be completely different.

s90
00:08:46.800 --> 00:08:50.000
So we have to be able to map that somehow.

s91
00:08:50.000 --> 00:08:54.000
And then when you get the query out of the LLM, it's untrusted.

s92
00:08:54.000 --> 00:08:57.360
It could have uh operations that are dangerous.

s93
00:08:57.360 --> 00:08:59.360
It could cross tenant boundaries.

s94
00:08:59.360 --> 00:09:05.600
And so you have to have some way of validating that the query is is uh correct and safe.

s95
00:09:05.600 --> 00:09:11.360
And then finally, uh, even though you do that validation, there could be errors.

s96
00:09:11.360 --> 00:09:15.760
Um, and when you actually do the query against the database, there could be errors there as well.

s97
00:09:15.760 --> 00:09:21.360
And so we can actually get the LLM to try to correct itself if it gets into that situation.

s98
00:09:21.360 --> 00:09:22.959
So this is what we're aiming for here.

s99
00:09:22.959 --> 00:09:25.920
We want to go to uh to this.

s100
00:09:25.920 --> 00:09:34.160
We want to be able to say something like open PR is by machine and then we want to get this kind of query on the back of that.

s101
00:09:34.160 --> 00:09:35.600
So, just take a sec here.

s102
00:09:35.600 --> 00:09:41.519
U this is the open- source repo that we're going to be building against.

s103
00:09:41.600 --> 00:09:46.959
And I'm going to jump right into what that looks like actually.

s104
00:09:48.640 --> 00:09:50.000
So, this is the repo here.

s105
00:09:50.000 --> 00:09:53.040
It's called the document query engine.

s106
00:09:53.040 --> 00:09:56.560
And I want to show you what it looks like when we just boot it up the first time.

s107
00:09:56.560 --> 00:10:02.480
So, I can say, you know, PR is by Peter at ground zero.

s108
00:10:02.480 --> 00:10:04.560
Oh, actually we're actually not running it.

s109
00:10:04.560 --> 00:10:07.120
So, I'll go back.

s110
00:10:07.440 --> 00:10:10.080
Let's run it first.

s111
00:10:15.519 --> 00:10:18.560
Now I can ask my question.

s112
00:10:19.200 --> 00:10:22.160
So, when you start it's going to have it's not going to be wired up at all.

s113
00:10:22.160 --> 00:10:23.120
It's going to produce nothing.

s114
00:10:23.120 --> 00:10:25.440
So, we're going to just gradually build this thing up here.

s115
00:10:25.440 --> 00:10:27.440
Okay.

s116
00:10:29.519 --> 00:10:33.680
First thing we're going to talk about is automatic schema discovery.

s117
00:10:33.680 --> 00:10:36.000
This is completely dynamic.

s118
00:10:36.000 --> 00:10:37.040
is schemalist.

s119
00:10:37.040 --> 00:10:42.079
Not and not all databases are schemalist, but the same kind of principle can apply.

s120
00:10:42.079 --> 00:10:45.839
You can dynamically discover the schema at runtime.

s121
00:10:45.839 --> 00:10:55.440
Um and so what what the technique involves is sampling a whole bunch of documents in the document store and then deriving a schema from that.

s122
00:10:55.440 --> 00:11:00.959
And uh you know I just want to show you like one tricky part of that which is inferring enums.

s123
00:11:00.959 --> 00:11:12.959
Um now the the the outcome from this is that you can use an LLM as a Thor hammer um and it can do a pretty good job but actually

s124
00:11:12.959 --> 00:11:17.839
you get uh really really far just using traditional procedural methods.

s125
00:11:17.839 --> 00:11:24.640
So you can you know don't always fall back to an LLM to do this kind of what looks like non-deterministic work.

s126
00:11:24.640 --> 00:11:28.640
This is actually quite deterministic this kind of thing.

s127
00:11:28.640 --> 00:11:33.920
So now that we've discovered schema, I'm just going to show you what that looks like in our demo.

s128
00:11:33.920 --> 00:11:34.880
Whoops.

s129
00:11:34.880 --> 00:11:44.160
First we have to level up to the next and then run.

s130
00:11:44.800 --> 00:11:53.839
So now we've discovered the schema from our um GitHub collection and this is just pull requests.

s131
00:11:53.839 --> 00:12:02.800
Okay, so it has not only the schema here but also um indexes that will help improve the performance of queries.

s132
00:12:05.680 --> 00:12:19.760
The next step is identity resolutions that thing I talked about about mapping you know users to uh real identities and um again this is this can be solved just using simple procedural tricks.

s133
00:12:19.760 --> 00:12:23.440
So um that that's actually more or less how we do it.

s134
00:12:23.440 --> 00:12:41.279
Uh and underneath the hooded unblocks when we take usernames or identities from different systems and map them across um you can do this kind of fuzzy matching and then in the end if you have to you can pass a list of names to an LLM and have it act as a tiebreaker.

s135
00:12:45.920 --> 00:12:51.519
And finally, uh, the last step before we can actually show the cool stuff is this.

s136
00:12:51.519 --> 00:12:53.839
It's the synthesis part.

s137
00:12:53.839 --> 00:12:57.120
And roughly speaking, this is actually the easiest part.

s138
00:12:57.120 --> 00:13:01.760
We just take the schema that we dynamically resolve from earlier.

s139
00:13:01.760 --> 00:13:13.440
We take the current date and time, uh, maybe some information about the user that's asking the question, so it can easily resolve questions like my PRs and not just Peter's PRs.

s140
00:13:13.440 --> 00:13:18.639
Um and then we pass it to the LLM and ask it to synthesize a query for us.

s141
00:13:18.639 --> 00:13:23.839
And the outcome uh will look something like this.

s142
00:13:24.480 --> 00:13:28.320
Let's get to this one.

s143
00:13:29.839 --> 00:13:35.920
So now I can go back to my engine and try to run that query again.

s144
00:13:37.600 --> 00:13:43.040
And it's going to pull the documents for me now and generate this query.

s145
00:13:45.360 --> 00:13:48.399
Now the next step I I was talking about was was validation.

s146
00:13:48.399 --> 00:13:50.320
This is actually the most important thing.

s147
00:13:50.320 --> 00:14:03.519
Um your your uh query could contain really dangerous operators like the function operator for example which would execute raw JavaScript um on the DB server which is definitely not ideal.

s148
00:14:03.519 --> 00:14:06.399
Um, but there could be other things that you want to do with this.

s149
00:14:06.399 --> 00:14:15.440
Like for example, you might have hidden fields that you don't want uh to be surfaced either to the LM or for the LM to to derive and execute.

s150
00:14:15.440 --> 00:14:31.040
If you have metadata fields, for example, that contain um tenant IDs and you want to isolate each query to a specific tenant, you're better off making sure that the LM doesn't interact with those fields at all and then just constraining it behind the scenes.

s151
00:14:31.040 --> 00:14:34.320
you can inject those fields underneath.

s152
00:14:36.959 --> 00:14:40.079
And that's kind of what that looks like.

s153
00:14:40.079 --> 00:14:46.079
So again, we're going to just quickly show the payoff for that and what that looks like.

s154
00:14:47.120 --> 00:14:50.639
Let's go up to our validation step.

s155
00:14:50.639 --> 00:14:52.560
Oops.

s156
00:14:52.560 --> 00:14:53.279
Run it again.

s157
00:14:53.279 --> 00:15:04.720
So if I now ask a question like this PRs with authentication in the title that is probably going to drive some kind of like regular expression thing and there we go.

s158
00:15:04.720 --> 00:15:05.680
Okay.

s159
00:15:05.680 --> 00:15:15.120
So uh our validation engine said sorry regular expressions aren't allowed you're getting bounced and now we're not allowed to do that.

s160
00:15:17.040 --> 00:15:18.959
So validation is cool.

s161
00:15:18.959 --> 00:15:22.000
Um but what do we do with that?

s162
00:15:22.000 --> 00:15:23.600
you know, we got an error.

s163
00:15:23.600 --> 00:15:33.680
Um, but what we can do instead is just take that error and just loop it back in to the LLM and have the LLM kind of self-correct.

s164
00:15:34.639 --> 00:15:36.399
And that's kind of what it that's all it is.

s165
00:15:36.399 --> 00:15:38.639
That's all it looks like from a technical standpoint.

s166
00:15:38.639 --> 00:15:43.839
It's just a loop with some sort of maximum retry.

s167
00:15:45.440 --> 00:15:56.639
And so if I try this again, bounce up to my retries and run that.

s168
00:15:57.440 --> 00:16:00.800
Now, if I try that query again.

s169
00:16:00.880 --> 00:16:03.440
Oops, that's the wrong one.

s170
00:16:03.440 --> 00:16:06.079
Let's try this one.

s171
00:16:12.800 --> 00:16:16.240
Now what you can see is that it actually took a couple attempts.

s172
00:16:16.240 --> 00:16:18.800
Um but it got there in the end.

s173
00:16:18.800 --> 00:16:25.040
However, um it constrained it down to use match and unfortunately this is an exact title match.

s174
00:16:25.040 --> 00:16:28.000
We don't have any PRs with that exact title.

s175
00:16:28.000 --> 00:16:35.680
So we need one more thing which is this full text search.

s176
00:16:35.680 --> 00:16:44.560
So now if we add full text search to our engine um hopefully the outcome that we get is something that looks a little bit like this.

s177
00:16:44.560 --> 00:16:47.839
So let's try it one more time.

s178
00:17:03.120 --> 00:17:05.120
Awesome.

s179
00:17:06.240 --> 00:17:07.280
So there you go.

s180
00:17:07.280 --> 00:17:09.760
We got we went from start to finish there.

s181
00:17:09.760 --> 00:17:10.880
This is all open source.

s182
00:17:10.880 --> 00:17:15.360
So you can take a look at how it's implemented and play around with it yourself.

s183
00:17:17.360 --> 00:17:18.640
And this is what we built.

s184
00:17:18.640 --> 00:17:24.240
Something that can effectively handle all of the uh other half of rag.

s185
00:17:24.240 --> 00:17:35.520
So, anything that has to do with uh structural based data, you can have filters, aggregations, it's time scoped, and it selfcorrects.

s186
00:17:35.520 --> 00:17:40.720
But what's really cool is that it also powers a chat agent.

s187
00:17:40.720 --> 00:17:56.400
And so if I go over here and I say something like um what authentication related things have we've been working on last week.

s188
00:18:00.559 --> 00:18:07.440
Now you get this whole thing that will show you exactly what it's doing, all the queries that it creates.

s189
00:18:08.000 --> 00:18:11.200
And now it has the data and it's just kind of reasoning about it behind the scenes.

s190
00:18:11.200 --> 00:18:13.679
And there we go.

s191
00:18:14.080 --> 00:18:16.400
Very cool.

s192
00:18:19.360 --> 00:18:20.240
So there you are.

s193
00:18:20.240 --> 00:18:25.520
Structured queryable context for agents and humans.

s194
00:18:25.520 --> 00:18:27.600
Now there's just one last thing.

s195
00:18:27.600 --> 00:18:33.679
Um on the description you probably saw that we were promising a context engine simulator.

s196
00:18:33.679 --> 00:18:44.720
If you don't want to wire up unblocked uh right away because you've got some constraints in your in your environment, um we have an open source project that will allow you to do this entirely

s197
00:18:44.720 --> 00:18:46.559
locally yourself.

s198
00:18:46.559 --> 00:18:58.240
And so the idea is uh it will create a context bundle for you on a taskbytask basis and then take that context bundle and run an AB test against the same task

s199
00:18:58.240 --> 00:19:04.240
and show you the benefits of having all of that context at your fingertips for your agents.

s200
00:19:04.240 --> 00:19:08.640
And that's the uh PR code for that if you want that.

s201
00:19:16.080 --> 00:19:18.160
All right.

s202
00:19:18.160 --> 00:19:24.400
And one last thing, if you want some coconuts, come by the booth.

s203
00:19:26.000 --> 00:19:28.240
Thank you.