WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=qdAkxLoYNI8

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/qdAkxLoYNI8.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:12.400 --> 00:00:15.120
What we do at Unblocks is we build a context engine.

s2
00:00:15.120 --> 00:00:18.640
I just want to do a quick sound check at the back to make sure everyone can hear me fine.

s3
00:00:18.640 --> 00:00:20.080
Can you guys Yeah, we're good.

s4
00:00:20.080 --> 00:00:21.279
Awesome.

s5
00:00:21.279 --> 00:00:30.480
So at a high level, a context engine delivers organizational context to both your human workers and now increasingly your agents.

s6
00:00:30.480 --> 00:00:30.800
Okay.

s7
00:00:30.800 --> 00:00:33.120
So why why is that important?

s8
00:00:33.120 --> 00:00:41.920
Before we go too deep on the mechanics of how a context engine works, I just want to talk briefly about the problem.

s9
00:00:42.800 --> 00:00:54.559
So, what we're going to do is we're going to hop into our time machines and we're going to travel back to the before times uh before agents and uh discuss a little bit about what we used to do as humans

s10
00:00:54.559 --> 00:00:58.079
uh before agents came into the picture.

s11
00:00:58.719 --> 00:01:02.879
And so for years um you were the context layer.

s12
00:01:02.879 --> 00:01:08.720
You um had to go and do things like this.

s13
00:01:08.720 --> 00:01:16.479
you had to find things you were looking for, trolled all over different data sources, different discussions taking place.

s14
00:01:16.479 --> 00:01:29.200
Um, and then through the codebase of course to try to build up tribal knowledge and, uh, throughout time as your code base progressed, um, you'd be, you know, fighting incidents and things like that.

s15
00:01:29.200 --> 00:01:42.400
And your organization over time builds up battle scars um from all these all these different things building code uh documenting architecture and and dealing with outages and things like that.

s16
00:01:42.720 --> 00:01:54.079
But now um we have a new problem because uh as we introduce agents to the picture um they suffer from all of these challenges except for one thing.

s17
00:01:54.079 --> 00:01:56.960
Agents are like new employees.

s18
00:01:56.960 --> 00:02:01.200
they reset their knowledge every time you start a new task.

s19
00:02:01.200 --> 00:02:02.399
Okay?

s20
00:02:02.399 --> 00:02:11.039
And so you can think of an agent like an expert software engineer um who's a new employee on boarding for the first time.

s21
00:02:11.039 --> 00:02:21.760
Every time they have to rediscover your code base, how your organization builds tests um and how they deploy software with each and every task.

s22
00:02:22.560 --> 00:02:30.080
Uh can I just uh put a put a show of hands for everyone that's seen this slide before by Vim?

s23
00:02:30.080 --> 00:02:37.360
So this is kind of like u this is a good way to view where people are on what we call like the AI maturity curve.

s24
00:02:37.360 --> 00:02:45.040
Um starting at the the far left uh this is kind of representative of autocomplete back in the GBT35 days.

s25
00:02:45.040 --> 00:02:47.760
You know remember co-pilot and things like that.

s26
00:02:47.760 --> 00:02:51.440
Um and then you know kind of move on to using cursor.

s27
00:02:51.440 --> 00:02:56.720
Um and then from there you're you're think you're talking about how you can start to solve the context problem.

s28
00:02:56.720 --> 00:03:00.239
So some people are building organizational wikis.

s29
00:03:00.239 --> 00:03:05.920
Just smile if if this is kind of um bringing up memories for you.

s30
00:03:05.920 --> 00:03:22.560
Um and then you know all these things are great except that uh how do you give agents access to this and what are the compounding problems that the scaling problems as you move forward well if you give MCP and skills to your agents

s31
00:03:22.560 --> 00:03:32.799
um to teach them how to navigate and build context and that's kind of where uh people are today most people they're at the sort of stage four to five

s32
00:03:32.799 --> 00:03:42.720
level okay and uh they understand that context is the bottleneck and they're trying to build solutions to solve it for their engineering teams.

s33
00:03:42.720 --> 00:03:48.080
So looking ahead uh to all the way to eight with software factories.

s34
00:03:48.080 --> 00:03:49.920
This is kind of where the puck is going.

s35
00:03:49.920 --> 00:03:58.080
I'm not sure if if folks were at the keynote this morning, but um it's it's all about like delivery of context and unknown and unknowns.

s36
00:03:58.080 --> 00:04:03.920
And this becomes increasingly important as people start thinking about full automation of agents.

s37
00:04:03.920 --> 00:04:06.959
they just can't operate without organizational context.

s38
00:04:06.959 --> 00:04:09.360
They get lost.

s39
00:04:11.120 --> 00:04:13.920
So, you know, like that's the real problem.

s40
00:04:13.920 --> 00:04:18.239
Access to information doesn't equal understanding.

s41
00:04:18.239 --> 00:04:27.440
Um I I know that folks are probably familiar with claude.md um and and uh and wiki layouts and all these things.

s42
00:04:27.440 --> 00:04:34.639
If you attach a wiki, it still doesn't tell the agent where the information is that it needs.

s43
00:04:34.639 --> 00:04:44.560
It can search for things in the wiki, but then what happens is it'll suffer from something that uh radiologists uh call satisfaction of search.

s44
00:04:44.560 --> 00:04:53.919
So, this is a term in radiology where you look at an X-ray and you're trying to find a region um that might be an indicator for cancer.

s45
00:04:53.919 --> 00:04:54.800
Okay?

s46
00:04:54.800 --> 00:05:08.800
And you discover like one indicator and if you stop there uh you might miss other important indicators that might you know lead to diagnosis of even more uh issues.

s47
00:05:08.800 --> 00:05:10.800
So this is what happens with agents.

s48
00:05:10.800 --> 00:05:15.759
They don't they they find something that they they think is correct and then they stop.

s49
00:05:15.759 --> 00:05:20.320
Um the the other thing about agents is that they don't distill understanding.

s50
00:05:20.320 --> 00:05:30.479
They can look around, they can find information, but they they don't understand how all the pieces fit together because without doing that leg work ahead of time.

s51
00:05:30.479 --> 00:05:40.880
Um, they don't understand how, you know, your dependencies interact with each other and how your architecture and sort of future planning is going to scope the work that it does next.

s52
00:05:40.880 --> 00:05:49.440
And so some some people will then ask, well, what if we just take the entire codebase and all of our architecture documents and just slam it into the context window.

s53
00:05:49.440 --> 00:05:54.000
Um, and then yes, maybe like your agents will reason about everything all at once.

s54
00:05:54.000 --> 00:05:56.800
And in practice, that that of course doesn't work.

s55
00:05:56.800 --> 00:06:05.280
Um, not just because you've got way more organizational context than can fit into a context window, even one that's a million tokens in size.

s56
00:06:05.280 --> 00:06:09.039
Um, but it it it causes the agent to get distracted.

s57
00:06:09.039 --> 00:06:13.440
When you're working on a task, you want task specific flow.

s58
00:06:13.440 --> 00:06:19.919
Um, and so your agents will get distracted easily if you give them things that cause them to look this way in that way.

s59
00:06:19.919 --> 00:06:23.120
Um, and it'll just waste tokens and time.

s60
00:06:23.120 --> 00:06:28.639
So, in this morning's keynote, um, Tariq from Claude Code mentioned unknown unknowns.

s61
00:06:28.639 --> 00:06:31.680
I just want to uh harp on that phrase again.

s62
00:06:31.680 --> 00:06:37.199
And it can be phrased a different way, which is finding the things that really matter.

s63
00:06:37.199 --> 00:06:41.039
And so this is what your agent can see at the top of the iceberg.

s64
00:06:41.039 --> 00:06:44.479
They can see the code and they can operate on the code.

s65
00:06:44.479 --> 00:06:59.120
What they don't see are things like the actual intent, the team conventions, past decisions, things that you've discussed in Slack, for example, uh architecture rationale, and so on.

s66
00:06:59.120 --> 00:07:05.280
And that's why your agents need a context engine to get real work done.

s67
00:07:05.280 --> 00:07:08.720
So, I'm going to now uh attempt a live demo.

s68
00:07:08.720 --> 00:07:11.360
And hopefully the demo gods are kind.

s69
00:07:11.360 --> 00:07:15.039
Um, so I want to pop back up conceptually.

s70
00:07:15.039 --> 00:07:16.560
Oops, I think I'm on the wrong tab.

s71
00:07:16.560 --> 00:07:19.919
We'll get to that one in a sec.

s72
00:07:20.560 --> 00:07:24.960
So for now, sorry about that.

s73
00:07:24.960 --> 00:07:28.319
And here we are.

s74
00:07:29.360 --> 00:07:37.680
So I'm going to ask a question as if I'm a, you know, I'm a human and I want to get some information about my codebase.

s75
00:07:38.000 --> 00:07:40.319
And, you know, the human layer hasn't gone away.

s76
00:07:40.319 --> 00:07:50.240
We talk about agents and their need for context, but um humans are still asking questions about the codebase and we need that level of understanding because ultimately the accountability

s77
00:07:50.240 --> 00:07:51.599
stops with us.

s78
00:07:51.599 --> 00:07:57.520
When you hit merge on a PR, you need to understand what it's doing um and you need to understand how the architecture works.

s79
00:07:57.520 --> 00:08:08.800
So this question I asked here um is about an internal component of our system called the source mark engine and you can see that it uh is able to articulate it fairly well.

s80
00:08:08.800 --> 00:08:11.120
um understands the architecture.

s81
00:08:11.120 --> 00:08:14.720
This this diagram here is uh is generated.

s82
00:08:14.720 --> 00:08:17.919
So it this diagram doesn't exist.

s83
00:08:17.919 --> 00:08:27.520
Um it just figures it out based on the um the way the code operates today and then some proposals for future architecture.

s84
00:08:27.680 --> 00:08:30.879
And then uh what's really important is that you show your work.

s85
00:08:30.879 --> 00:08:45.839
This is a trust building thing more than anything, but it allows people to see if um if the answer is maybe not entirely correct, then you can in look into the uh the knowledge base that you have and make corrections.

s86
00:08:45.839 --> 00:08:49.600
Increasingly agents are doing this for you.

s87
00:08:49.680 --> 00:09:00.720
So now um what I want to show you is another place where humans spend their time which is in Slack and this is where a lot of the decisions get made of course.

s88
00:09:00.720 --> 00:09:04.080
So I can do something like this.

s89
00:09:04.080 --> 00:09:11.760
And uh unblocked will sit and kind of listen for things that are things that can chime in on when it provides a high degree of Oh, sorry.

s90
00:09:11.760 --> 00:09:15.680
We went to the wrong You guys can't see that.

s91
00:09:15.680 --> 00:09:18.160
Thank you, Claire.

s92
00:09:18.560 --> 00:09:20.080
Oh, come on down.

s93
00:09:20.080 --> 00:09:22.560
Let's see if I can bring it up.

s94
00:09:22.560 --> 00:09:24.959
There we go.

s95
00:09:24.959 --> 00:09:26.080
Perfect.

s96
00:09:26.080 --> 00:09:31.760
So I can ask questions like this in unblocked and if it thinks it can chime in on the answer then it will chime in.

s97
00:09:31.760 --> 00:09:46.000
Otherwise I can just um address unblocked directly and ask the same question and when it thinks that it has an answer to give then it will give an answer

s98
00:09:46.000 --> 00:09:51.760
and so we can get um quite a bit of interesting content there from unblocked.

s99
00:09:51.760 --> 00:09:52.160
Thank you.

s100
00:09:52.160 --> 00:09:54.160
Unblocked.

s101
00:09:55.360 --> 00:10:01.440
I'm going to switch up and show you the the really interesting thing which is the agents.

s102
00:10:01.440 --> 00:10:01.839
Okay.

s103
00:10:01.839 --> 00:10:10.240
So, um in in that question, the source mark engine, I'm not sure if people picked up, but there was a little thing at the bottom there that said, you know, there's some optimization

s104
00:10:10.240 --> 00:10:11.600
opportunities.

s105
00:10:11.600 --> 00:10:25.680
Um so what I did here is I went into claw code and I asked it um without using unblocked to um uh generate uh a plan to optimize the source mark calculator and it did that

s106
00:10:25.680 --> 00:10:31.600
and it happily went and you know searched through the code and and tried to figure out how the algorithm works and so on.

s107
00:10:31.600 --> 00:10:39.920
Um and it it reached a conclusion that's great you know it does a pretty good job um but you know it maybe could do a little bit better.

s108
00:10:39.920 --> 00:10:54.480
So, I asked that question again uh using unblock this time and it it really kind of nails the the nuances because it picks up on the the uh PRs that we um where we discussed

s109
00:10:54.480 --> 00:10:57.040
future possibilities for improvement.

s110
00:10:57.040 --> 00:11:17.519
um some Slack conversations that we had and uh of course you know notion and architecture documents and it shows its work and this is really important because um all of these things here the sources come back to Claude and then Claude knows exactly where to jump to next if it needs to elaborate on that context.

s111
00:11:17.519 --> 00:11:20.640
And so I just want to show you what the impact of that is.

s112
00:11:20.640 --> 00:11:24.480
So if I um Whoops.

s113
00:11:24.480 --> 00:11:25.279
Thank you.

s114
00:11:25.279 --> 00:11:32.959
If I pull up usage here, you can see that with unblocked, uh, the total cost was, you know, subd dollar to create the plan.

s115
00:11:32.959 --> 00:11:34.320
Uh, took about a minute.

s116
00:11:34.320 --> 00:11:37.279
Ignore the wall clock time because I've had this open for about an hour.

s117
00:11:37.279 --> 00:11:39.839
But, um, it's about a minute.

s118
00:11:39.839 --> 00:11:47.120
And then if I look at um the usage without unblocked, you can see that it's about 2 minutes.

s119
00:11:47.120 --> 00:11:50.399
And and and it costs more to generate all that context.

s120
00:11:50.399 --> 00:11:53.200
Now, the reason that happens is because it has to do more work.

s121
00:11:53.200 --> 00:11:54.320
It has to look around.

s122
00:11:54.320 --> 00:11:55.920
has to discover things.

s123
00:11:55.920 --> 00:12:02.800
Um, and this compounds, not only does it have to do more work to discover things, it doesn't discover the right things.

s124
00:12:02.800 --> 00:12:09.200
So, when you get further down in your execution, it may be operating on the wrong plan or the wrong assumptions.

s125
00:12:09.200 --> 00:12:12.639
And then you have to go back and you have to loop over and over again.

s126
00:12:12.639 --> 00:12:17.839
So, the real value of a context engine is not like the upfront cost on these short tasks.

s127
00:12:17.839 --> 00:12:19.920
It's the compounding effect.

s128
00:12:19.920 --> 00:12:32.320
Um the the other Tariq from Sonar mentioned this in the keynote this morning and it's true like the loops compound and you have to be like um uh efficient the entire way through with your context.

s129
00:12:32.320 --> 00:12:39.920
I'm just going to jump back to Safari and I'm going to point out um some really interesting things.

s130
00:12:39.920 --> 00:12:44.639
So we also have a a code review agent.

s131
00:12:44.639 --> 00:12:53.040
And when we say um you know organizational context, we're talking about more than just the underlying data.

s132
00:12:53.040 --> 00:12:55.839
Uh we're talking about real intelligence.

s133
00:12:55.839 --> 00:13:07.040
So what unblock does is it looks at um not like it looks at pull request data and there are other data sources for this and it generates a series of best practices

s134
00:13:07.040 --> 00:13:09.680
that help align agents to your codebase.

s135
00:13:09.680 --> 00:13:14.240
But we thought that this would be really helpful to surface for the review agent as well.

s136
00:13:14.240 --> 00:13:22.240
So what you can see here is um it unblock chimed in and then Richie here said, "Oh, that's cool.

s137
00:13:22.240 --> 00:13:23.279
That's something I would say."

s138
00:13:23.279 --> 00:13:26.320
And that's because that actually was something he said.

s139
00:13:26.320 --> 00:13:29.040
So it surfaced the uh the previous comments.

s140
00:13:29.040 --> 00:13:40.320
Richie's one of the senior engineers and we use the sort of seniority or expertise as a signal um to boost uh comments that are important.

s141
00:13:40.320 --> 00:13:42.079
Okay.

s142
00:13:42.079 --> 00:13:56.800
So another uh interesting interaction by Richie, he uh discovered that the number of code review issues that were being surfaced dropped uh precipitously and he was debugging it with unblocked.

s143
00:13:56.800 --> 00:14:02.880
Um he got all the way to the bottom and realized what roughly what the problem was and then asked unblocked to fix it.

s144
00:14:02.880 --> 00:14:08.560
Now this this is something that we have internally um you know that we're experimenting with.

s145
00:14:08.560 --> 00:14:13.120
Um, so unblocked uh can run as an agent in the cloud.

s146
00:14:13.120 --> 00:14:23.040
Um, but what's really cool about this is that it has all your organizational context at its fingertips and the results are are pretty magical.

s147
00:14:23.040 --> 00:14:35.199
So it can do things like generate this PR um, and then what you'll see here is that not only does it generate the fix, it also is able to relate it to the all the conversations that were happening.

s148
00:14:35.199 --> 00:14:38.639
So this PR was created because and you read that context thing.

s149
00:14:38.639 --> 00:14:39.920
It's mind-blowing.

s150
00:14:39.920 --> 00:14:49.279
After this PR, we switched to uh Claude 48 and it dropped a ton in issues because of the behavior is quite a bit different.

s151
00:14:49.279 --> 00:14:52.320
So then it said Richie directly correlated the drop.

s152
00:14:52.320 --> 00:14:53.279
Now what's this thing here?

s153
00:14:53.279 --> 00:14:54.399
Let's click on it.

s154
00:14:54.399 --> 00:14:56.320
It is a Slack conversation.

s155
00:14:56.320 --> 00:15:06.480
So, it found the Slack conversation, correlated all of that, you know, past history back again, and then we ended up with a with a final PR.

s156
00:15:08.560 --> 00:15:11.360
So, um, I'm going to I've got only a few minutes left.

s157
00:15:11.360 --> 00:15:13.279
I'm just going to close this out really quickly.

s158
00:15:13.279 --> 00:15:18.560
We have a uh a couple of open- source projects that are kind of interesting if people want to play with them.

s159
00:15:18.560 --> 00:15:20.320
One is the document query engine.

s160
00:15:20.320 --> 00:15:24.480
That was, uh, something that I talked about on Monday in my workshop.

s161
00:15:24.480 --> 00:15:30.160
Um, I may uh talk about it again tomorrow, but I just want to give folks a sense of what this thing does.

s162
00:15:30.160 --> 00:15:32.560
Um, whoops.

s163
00:15:32.639 --> 00:15:37.440
If you want to play with it, it's open source, so you can just download it and have it go.

s164
00:15:37.440 --> 00:15:50.639
It basically runs over your um uh GitHub repository, ingests uh your your historical pull requests, and then uh synthesizes a schema based on the documents that it can sample.

s165
00:15:50.639 --> 00:15:57.920
Um and then from there you can issue any kind of queries that you like and get all kinds of insights out of it through the agent chat.

s166
00:15:57.920 --> 00:16:00.639
You can ask all kinds of questions.

s167
00:16:00.639 --> 00:16:03.680
Um and then lastly the engineering social graph.

s168
00:16:03.680 --> 00:16:11.120
So this is the thing that I was talking about earlier that helps us pin down expertise and team relationships.

s169
00:16:11.120 --> 00:16:17.199
Um so what you can see here is this sort of like the rough breakdown of our team structure at Unblocked.

s170
00:16:17.199 --> 00:16:19.600
As you can see we're a fairly small team.

s171
00:16:19.600 --> 00:16:30.639
Um and so we've got these um uh these clusters of people and how they relate to each other indicates the kind of um review relationships that they have.

s172
00:16:30.639 --> 00:16:35.040
So these are you know these lines show like we review each other's code.

s173
00:16:35.040 --> 00:16:44.079
Um we can then cluster that and generate team labels for that or show the coverage across your codebase.

s174
00:16:44.079 --> 00:16:44.800
This is really cool.

s175
00:16:44.800 --> 00:16:49.360
you can see kind of where the holes are, where you might be lacking expert coverage.

s176
00:16:49.360 --> 00:16:54.800
Um, and that's exactly what we use within the context engine itself.

s177
00:16:54.880 --> 00:17:09.039
All right, one last thing we have uh for those that want a taste of what a context engine can do but don't want to sign up for unblocked right away.

s178
00:17:09.039 --> 00:17:22.559
Um you can use uh something that we call the context engine simulator which will basically build up a context behind the scenes on a per task basis and then use that context

s179
00:17:22.559 --> 00:17:24.400
uh to to drive the task.

s180
00:17:24.400 --> 00:17:30.799
It'll do it with context and without context so that you can see what the differences might be.

s181
00:17:30.799 --> 00:17:36.320
This is a QR code for that if you want to just take a quick snap.

s182
00:17:38.240 --> 00:17:38.880
Awesome.

s183
00:17:38.880 --> 00:17:45.200
And I'll just land on a quote from one of our customers.

s184
00:17:45.200 --> 00:17:50.240
50% fewer tokens, faster triage, better answers.

s185
00:17:50.240 --> 00:17:54.160
And that's exactly what a context engine can do.

s186
00:17:54.480 --> 00:17:57.200
One last shout out um before we end.

s187
00:17:57.200 --> 00:18:04.480
My colleague Brandon is giving a talk in 10 minutes uh at room 2020.

s188
00:18:04.480 --> 00:18:10.080
um he's going to speak to in a lot more detail about some of the higher level things that context engines can do.

s189
00:18:10.080 --> 00:18:14.400
I'm going to run over there right after this and I think all of you should follow me.

s190
00:18:14.400 --> 00:18:15.280
Awesome.

s191
00:18:15.280 --> 00:18:18.559
Oh, and don't forget to get a coconut.

s192
00:18:33.512 --> 00:18:35.512
[music]