WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=HvMyYLTfvhg

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/HvMyYLTfvhg.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:13.440 --> 00:00:14.360
Hi all.

s3
00:00:14.360 --> 00:00:15.320
Uh my name is Jeff.

s4
00:00:15.320 --> 00:00:22.320
I'm a founding engineer at Unblock, and I'm here to talk to you about how building agents has actually gotten pretty easy.

s5
00:00:22.320 --> 00:00:26.880
But unfortunately, they still get things confidently wrong.

s6
00:00:26.920 --> 00:00:33.520
So, 6 months ago, it required a team's effort and basically a quarter to build out an agent.

s7
00:00:33.520 --> 00:00:37.120
Um an agent is more than just models and tools.

s8
00:00:37.120 --> 00:00:43.320
It's the models, the tools, and everything required to build out a production service.

s9
00:00:43.720 --> 00:00:49.960
Here are some examples of the different systems that were necessary in order to build something out.

s10
00:00:49.960 --> 00:00:54.560
Each one of these was basically its own company or at least a company function.

s11
00:00:54.560 --> 00:00:59.200
Not going to go through each one of these, but you know, a few that stood out to me.

s12
00:00:59.200 --> 00:01:02.640
First one, checkpoint and state persistence.

s13
00:01:02.640 --> 00:01:06.400
Agent runs, they're typically long-lived and stateful.

s14
00:01:06.400 --> 00:01:12.800
Um unfortunately, uh infrastructure itself though, those that's ephemeral.

s15
00:01:12.800 --> 00:01:16.520
Crashing without durability can actually lead to a lot of state loss.

s16
00:01:16.520 --> 00:01:26.040
And that state kind of includes things like message history, tool calls, as well as, you know, where you are in the loop.

s17
00:01:26.040 --> 00:01:29.400
Without these things, you can't resume the session.

s18
00:01:29.400 --> 00:01:32.400
Uh one option is, you know, maybe you want to restart the session.

s19
00:01:32.400 --> 00:01:36.000
Unfortunately, that's actually quite expensive as well.

s20
00:01:36.000 --> 00:01:43.920
Uh you lose out on all the tokens that you'd originally used, um as well as, you know, latency.

s21
00:01:43.920 --> 00:01:47.360
Uh from a user experience standpoint, you've already triggered that session.

s22
00:01:47.360 --> 00:01:50.120
Now you have to wait for the whole thing to go again.

s23
00:01:50.120 --> 00:01:52.320
And lastly, side effects.

s24
00:01:52.320 --> 00:01:57.800
Your agent might have performed some side effects, and now there's a chance of those doubling up.

s25
00:01:58.000 --> 00:02:01.760
So, next thing, sandbox infrastructure, right?

s26
00:02:01.760 --> 00:02:07.760
So, as we all know, we're running more and more agent-generated code as well as third-party code.

s27
00:02:07.760 --> 00:02:13.200
This gets all run on your infrastructure, and due to that, there are some complexities.

s28
00:02:13.200 --> 00:02:28.880
Uh because of that, we want to introduce isolated sandboxes, which help prevent uh unnecessary reads of environment secrets, unnecessary network access, you know, just in general, we don't want to take down the shared host.

s29
00:02:29.959 --> 00:02:31.959
And then, observability.

s30
00:02:31.959 --> 00:02:35.760
How do we answer the question, "Where did this fail?"

s31
00:02:35.760 --> 00:02:43.480
Typically, this includes tracking logs and traces from across half a dozen systems.

s32
00:02:43.800 --> 00:02:48.880
Everything I've mentioned here, none of this actually improves an agent's capabilities.

s33
00:02:48.880 --> 00:02:55.280
They're all taxes one has to pay in order to get an agent out there to play the game.

s34
00:02:56.800 --> 00:02:59.880
Thankfully, things have changed quite a bit.

s35
00:02:59.880 --> 00:03:13.600
Um the whole ecosystem has matured quite a bit, and cloud infrastructure players such as Cloudflare, uh Vercel, AWS, they've gone and taken some of that complexity away and built primitives

s36
00:03:13.600 --> 00:03:25.600
that these frameworks, Flu, Vercel E Maestra, with these together, you know, they've taken a lot of complexity away, and you can focus more on building the actual agent itself.

s37
00:03:25.600 --> 00:03:31.560
The core logic that actually helps you and your team and your customers.

s38
00:03:33.560 --> 00:03:36.040
So, here's an example of one.

s39
00:03:36.040 --> 00:03:45.840
Uh I played around with Flu and Cloudflare, and as you can see on the left-hand side, you know, we basically handle everything as mentioned before.

s40
00:03:45.840 --> 00:03:53.640
So, the primitives plus the framework lead to a situation where it's actually not that much code to define an agent.

s41
00:03:53.640 --> 00:03:57.800
Uh one of the things I was shocked at when I first took a look at the documentation.

s42
00:03:57.800 --> 00:04:04.000
To get in the details, all you really have to do when defining agent is A, deciding which model you want to use.

s43
00:04:04.000 --> 00:04:07.360
B, the instructions or, you know, the system prompt.

s44
00:04:07.360 --> 00:04:12.640
C, the tools that you want to ask the agent to have access to.

s45
00:04:12.640 --> 00:04:15.120
Skills, the things I can do.

s46
00:04:15.120 --> 00:04:19.239
As well as the sandbox location, where things are being run.

s47
00:04:19.239 --> 00:04:31.360
So, uh to give you an example of this, I've actually gone and built out a issue enrichment system specifically for Linear.

s48
00:04:31.360 --> 00:04:41.520
So, what this does is, given a Linear ticket and access to your code repository, it'll go out, you know, fetch the Linear ticket, determine whether or not it's a feature

s49
00:04:41.520 --> 00:04:43.000
or a bug.

s50
00:04:43.000 --> 00:04:51.240
From there, it'll do some code searching, provide all that context to the agent, and then come up with a plan of next steps.

s51
00:04:51.240 --> 00:04:59.360
On the left-hand side here, this is a issue that one of my colleagues, uh smart engineer, had posted, I think, a month ago.

s52
00:04:59.360 --> 00:05:06.680
Uh to summarize it, what had happened was, we had some pretty serious degradation in our agentic QA pipeline.

s53
00:05:06.680 --> 00:05:13.240
Time to first character was taking 3 to 4 seconds when it should realistically be in the hundreds of milliseconds.

s54
00:05:13.240 --> 00:05:18.240
So, let's see what happens when, you know, we put this through the system.

s55
00:05:20.040 --> 00:05:24.360
So, as you'll see here, I've set up the agent to go fetch the agent.

s56
00:05:24.360 --> 00:05:31.160
I've given it the skills and tools to actually go and fetch the code, search the code, and query against that.

s57
00:05:31.160 --> 00:05:36.640
That's being passed back to the agent, which is doing some reasoning against that right now.

s58
00:05:36.800 --> 00:05:38.240
And then, wait a little bit.

s59
00:05:38.240 --> 00:05:42.600
At this point, we've updated the Linear issue ticket.

s60
00:05:42.600 --> 00:05:47.080
The recommendation here is to re-enable our async dispatch, which makes sense.

s61
00:05:47.080 --> 00:05:52.760
It allows us to run a lot more of our QE pipeline in parallel on a single machine.

s62
00:05:52.760 --> 00:05:54.160
Sounds great, right?

s63
00:05:54.160 --> 00:05:57.400
Unfortunately, uh this is wrong.

s64
00:05:57.400 --> 00:06:07.640
This had actually caused an outage a few days ago in one of our uh support engineers had explicitly disabled this uh before this ticket was uh shown.

s65
00:06:07.640 --> 00:06:11.880
So, where did things go wrong?

s66
00:06:11.880 --> 00:06:15.400
Why was the uh you know, why did I get it wrong?

s67
00:06:15.400 --> 00:06:18.520
The agent I had written, it didn't have a full picture.

s68
00:06:18.520 --> 00:06:31.040
It was missing the context from the Slack discussion that happened after the issue where the engineers came together, uh went through the actual outage, what went wrong, what was the fix, and the next steps.

s69
00:06:31.040 --> 00:06:36.400
It also was missing the postmortem uh linear ticket, which came as a result of that.

s70
00:06:36.400 --> 00:06:40.840
In general, it had a narrow understanding of the problem.

s71
00:06:41.000 --> 00:06:49.040
This concept of missing knowledge and intent that's stored across an organization and different systems is something that comes back and back again.

s72
00:06:49.040 --> 00:07:00.080
And since this was deployed as a background agent, this is going to make that mistake silently in the background, misinforming both my teammates and potentially other agents.

s73
00:07:00.080 --> 00:07:03.720
So, I guess the next question is, why don't we run into this locally?

s74
00:07:03.720 --> 00:07:08.360
You know, we all use agents locally, we don't necessarily run into these issues.

s75
00:07:08.360 --> 00:07:14.200
Well, you, the human, the engineers, we currently act as that context layer.

s76
00:07:14.200 --> 00:07:23.520
When working with an agent, you know, you're there to ask questions, catch any errors, and supply the missing facts on every single turn.

s77
00:07:23.520 --> 00:07:30.360
A person knew why the code is the way it is, what broke last time, and what we've decided to do about it.

s78
00:07:30.360 --> 00:07:33.280
The agent, though, it only has what's on the right-hand side, right?

s79
00:07:33.280 --> 00:07:40.919
It has instructions, the tools and skills we specifically gave it, the code, as well as the ticket in front of it.

s80
00:07:40.919 --> 00:07:47.840
When an agent is in the loop, well, sorry, when a human is in the loop with the agent, we're there to catch the steer.

s81
00:07:47.840 --> 00:07:50.600
Ultimately, we're there to babysit the agent.

s82
00:07:50.600 --> 00:08:02.000
But as agents have gotten trivially easy to deploy as I showed earlier with Flu Cloudflare, the without the human in the loop, this issue becomes more and more prevalent.

s83
00:08:02.000 --> 00:08:05.040
This missing context becomes a sign of failure.

s84
00:08:05.040 --> 00:08:10.760
You know, all that intuition and knowledge that we've had as humans needs to be replaced.

s85
00:08:10.760 --> 00:08:14.160
Something needs to carry the load.

s86
00:08:14.680 --> 00:08:17.600
So, that thing, that's a context engine.

s87
00:08:17.600 --> 00:08:25.680
A context engine is a system that provides task-relevant information based on who you are and what matters.

s88
00:08:25.680 --> 00:08:29.480
It also resolves all the conflicts across multiple data sets.

s89
00:08:29.480 --> 00:08:36.719
It understands your access rules or the agent's access rules and only uh respects that and only provides information that's relevant.

s90
00:08:36.719 --> 00:08:46.960
And most importantly, it delivers a synthesized understanding that an agent can act on, not just a list of documents that I have to reason upon itself.

s91
00:08:47.680 --> 00:08:50.520
So, how does this context engine work?

s92
00:08:50.520 --> 00:08:52.480
Well, let's take a step back.

s93
00:08:52.480 --> 00:08:55.160
What does an agent actually need?

s94
00:08:55.160 --> 00:09:00.120
An agent needs Clearly, it needs context outside of just your source code.

s95
00:09:00.120 --> 00:09:03.640
Think about everything that you need to work day-to-day.

s96
00:09:03.640 --> 00:09:05.040
It's not just the code.

s97
00:09:05.040 --> 00:09:13.440
It's, you know, the Slack discussions where decisions are made, the documentation where we show all the best practices.

s98
00:09:13.440 --> 00:09:17.360
All that is important to your day-to-day process, and that's true for your agent as well.

s99
00:09:17.360 --> 00:09:19.440
So, what we do here is we connect everything.

s100
00:09:19.440 --> 00:09:23.920
The docs, code, tickets, conversations.

s101
00:09:23.920 --> 00:09:27.600
We then build a model of your organization, of your system.

s102
00:09:27.600 --> 00:09:32.640
And we piece how all these things work together and make it generally available to your agents.

s103
00:09:32.640 --> 00:09:40.760
From that model, the agents are only provided a a of that data, which has been reconciled, ranked, and scoped to your permissions.

s104
00:09:40.760 --> 00:09:45.320
Scattered context comes in, grounded context comes out.

s105
00:09:45.920 --> 00:09:49.680
The obvious next question is, why can't we just do this with MCP, right?

s106
00:09:49.680 --> 00:09:57.480
You could connect a Slack MCP, a Linear MCP, a GitHub MCP, and with that, all that data is accessible.

s107
00:09:57.480 --> 00:10:02.160
MCP is great at access, but access isn't understanding.

s108
00:10:02.160 --> 00:10:09.960
An MCP hands the agent the raw results, and you know, you're now dependent on that agent to actually decide what to believe in.

s109
00:10:09.960 --> 00:10:17.840
You end up flooding the agent with irrelevant data, filling up the context window, and you know, overall context costs just go up.

s110
00:10:17.840 --> 00:10:21.040
It also leaves the local agent to handle conflicts in data.

s111
00:10:21.040 --> 00:10:25.880
You know, your Linear MCP and your Slack MCP may come back with different results.

s112
00:10:25.880 --> 00:10:30.760
You're just leaving the agent to make that decision somewhat ad hoc at the moment.

s113
00:10:32.280 --> 00:10:35.200
So, back to the original problem I had earlier.

s114
00:10:35.200 --> 00:10:40.800
This is the same file, same engine, but now we've connected the context agent.

s115
00:10:40.800 --> 00:10:49.560
Uh what we do here is is we're currently prompting Unblock to do some research on the ticket and provide that context to the agent.

s116
00:10:49.560 --> 00:10:52.839
So, let's see that in action.

s117
00:10:54.880 --> 00:10:57.600
Sorry about that.

s118
00:11:01.960 --> 00:11:02.960
So, here we go.

s119
00:11:02.960 --> 00:11:04.200
Uh we're doing the very similar thing.

s120
00:11:04.200 --> 00:11:06.160
We're fetching the Linear ticket.

s121
00:11:06.160 --> 00:11:09.680
But, you'll notice here that we're actually calling the Unblock context engine.

s122
00:11:09.680 --> 00:11:18.839
And what's done here is actually it's found the relevant Linear postmortem, as well as a Slack conversation where we've had the entire discussion between the engineering teams.

s123
00:11:18.839 --> 00:11:26.040
And as part of that, we've returned a understanding, and that's now been provided to the agent as a summary.

s124
00:11:26.040 --> 00:11:30.720
So, the agent no longer has to actually reason from those documents.

s125
00:11:31.000 --> 00:11:41.120
Uh at this point, you'll notice here the agent now has been updated.

s126
00:11:41.120 --> 00:11:49.440
Uh the recommendation has gone from breaking and causing another issue to actually preventing a another outage.

s127
00:11:53.360 --> 00:11:59.440
So, the example I've shown here is issue ticket management, but this context layer can actually go a lot further.

s128
00:11:59.440 --> 00:12:02.120
Uh for example, coding.

s129
00:12:02.120 --> 00:12:05.760
Everyone here does uh coding with uh cloud code or cortex.

s130
00:12:05.760 --> 00:12:13.400
Using an Unblocked context engine to actually hydrate the agent plan goes a long way in terms of saving context and tokens.

s131
00:12:13.400 --> 00:12:15.400
Uh code review.

s132
00:12:15.400 --> 00:12:19.520
It makes the PRs look as if they've been reviewed by an expert on your team.

s133
00:12:19.520 --> 00:12:21.160
Who doesn't like that?

s134
00:12:21.160 --> 00:12:27.720
As well as surfacing the correct answers to your customer success team as well as sales.

s135
00:12:27.720 --> 00:12:36.000
In general, there are many instances where you might want an agent to have institutional and tribal knowledge of your organization.

s136
00:12:37.040 --> 00:12:38.280
Just want to leave you on this.

s137
00:12:38.280 --> 00:12:42.000
I think this quote encapsulates what we're trying to solve at Unblocked.

s138
00:12:42.000 --> 00:12:45.920
The gap isn't intelligence, it's context.

s139
00:12:46.440 --> 00:12:48.120
So, thank you.

s140
00:12:48.120 --> 00:12:53.839
Uh I'll be at booth P16 along with the rest of my team if you guys have any questions.

s141
00:12:53.839 --> 00:13:03.520
There will be additional breakout sessions later tomorrow, I believe, that goes a lot more in depth about actually how the context engine works and you know, how you can benefit from that.

s142
00:13:03.520 --> 00:13:04.699
Cheers.

s143
00:13:04.699 --> 00:13:06.699
[applause]