Your agents lack context: Here's how to fix "You're absolutely right!" — Brandon Waselnuk, Unblocked
https://www.youtube.com/watch?v=KcVkq5L-0f0
AI Engineer · 14 min · 169 sentences · sentence-level transcript from YouTube's caption track; each line begins at the time shown

[00:01] [music]
[00:12] Good afternoon.
[00:13] I hope you're all having a lovely day here at AIE.
[00:16] We've had great weather, though the UV has been like nine.
[00:19] So, hopefully you put your sunscreen on your being appropriate adults.
[00:22] I'm here to talk to you about context engineering, and I have the good fortune of following AJ from LinkedIn because he talked a lot about the system that we actually design and sell to other solutions.
[00:30] And I'm going to give you a bunch of open source tools.
[00:32] So, if you watch that last talk just before me, you're going to get a bunch of tool chance you can go mess around yourself, and I'll teach you a bunch of techniques today.
[00:39] The goal, of course, is to fix your absolutely right.
[00:43] I think they've taken that out of the prompts now, so it just says you're right or other things, but I'm sure you've all been there.
[00:49] So, I'm Brandon.
[00:50] I work at Unblocked.
[00:51] Uh yes, I have a coconut.
[00:53] We've been giving these away for fresh context, fresh fresh coconuts.
[00:57] But, the thing that I want to talk to you about is with these models, especially with Meth O'Clock models, I think Fable 5's coming back today, so they say.
[01:04] You can watch my Grain Call recording try to book this.
[01:07] We'll ignore it.
[01:09] But, what I want you to do is to think about the fact that with these tools, AI-generated code should feel like it was written by someone who's been on your team for years.
[01:21] So, to get in the right headspace, for years you have to consider that you have been the context engine.
[01:27] How did you do that?
[01:29] You built context by going to work and asking questions, shipping PRs and getting them rejected, going to meetings, and all this slowly over time built up the engine that is your brain.
[01:40] You understand how it works here.
[01:42] You know how stuff gets shipped.
[01:43] You were on call that night when you took prod down and why that happened.
[01:48] The problem is that these agents have this exact same problem.
[01:52] Every time you create a new terminal session with an agent in it, it's very intelligent, but it doesn't have any context on how your company operates.
[01:59] So, it needs to get that somehow.
[02:02] The problem is as you move these agents up in scale, that cost compounds if you get it incorrect at the beginning.
[02:09] The leverage of context and content We're just going to fix this cuz I think people want to take some photos.
[02:17] Perfect.
[02:19] That context issue will compound.
[02:21] So, at the far left, we all remember the age-old time of 2 years ago where we had tab complete models that were pretty cool.
[02:28] What happened is it popped up and said, "Hey, do you want to tab this?"
[02:31] And quickly in your head with your context engine, you go, "No, that's bad."
[02:34] Or you went, "Oh, sweet."
[02:35] You hit tab.
[02:36] Nice.
[02:36] As we move along the agentic kind of adoption curve, what happens is you are moving into more situations in which you have agents running without a human in the loop, or at least you wish you didn't have to be in the loop.
[02:48] What they need is some way to be able to ask the questions they need when they hit walls in order to write code or solve or basically fix the issue and ultimately
[02:57] output code that's mergeable into your code base, especially with many people here who actually work in brownfield code bases that have been around for a long time that run real revenue across them, not just greenfield fun projects.
[03:10] So, that cost of bad context compounding at the beginning is cheap.
[03:14] If you think like shift left, finding a defect or a bug, you want to find it as early as possible.
[03:19] It's the same with context.
[03:21] Cuz as you move across, you get into doom loops.
[03:24] You usually ask your to do something.
[03:25] It's like, "Hey, I did it."
[03:26] And you're like, "No, man."
[03:28] And then you correct and correct and correct.
[03:30] That's wasted search tokens.
[03:31] It's also wasted rework time.
[03:34] And that is not acceptable with the tokenomics we have coming.
[03:38] And then as you move into parallel agents, etc., you start hitting a review tax.
[03:41] So, these AI code reviewers we're trying to use, but again, key context is important there so that those code reviews are able to basically understand how the operations of the business are so it knows the business logic and more.
[03:54] And then finally, if your hope is to move all the way out of the loop, you're like background agents, get it done, make no mistakes, you really need to make sure that you have a context engine so those agents can query it and get all the answers they need so they can keep operating
[04:07] in an effective way.
[04:10] There are some common approaches that don't work.
[04:13] They're basically like a local maxima.
[04:15] Two of the ones we see the most with our hundreds of enterprise clients and mid-market size businesses is the curated context trap.
[04:23] If you've ever sat down and taken a virtual file system or maybe a local file system, you put some markdown files in it and you're like, here's all the context of this project, it's how it works.
[04:31] You then allow your agent to grep over that and it gets a bunch of good data and then it will perform better.
[04:36] The issue is first, now you have to distribute that so maybe you throw it up in a GitHub and your team can grab it.
[04:42] But then the next is that repo is going to rot just like all the other docs you wrote down and then who at your org is the omnipotent one who has the taste to curate this file or repo
[04:53] for literally everyone in the org.
[04:54] So you start to hit these issues.
[04:57] The next is the MCP plateau.
[04:59] This one is pretty clear.
[05:01] We have MCPs, they're great.
[05:03] You can give it to your agent and now it can basically get information from another source system.
[05:07] The problem is, of course, based on how you write the server description, the tool descriptions, your agent may never call it even though it should have.
[05:16] Or if it does, there's a known bias called the satisfaction of search bias.
[05:20] What that means is the agent, when it finds the first piece of information that it thinks is correct, it goes, "Oh, I have what I need."
[05:26] and it proceeds.
[05:28] In most organizations, there's a Slack conversation from last night that says you should be doing A instead of doing B and the agent will never find it if it found some architecture record first.
[05:38] So it doesn't actually consider all of the context.
[05:42] The problem here is access to information is not understanding.
[05:47] So to deliver understanding to a model, you have to do other techniques.
[05:52] What I'm basically trying to say is what your agent can't see is everything below the waterline.
[05:57] It can 100% get code that compiles, but that code that compiles is taking down prod and you have a P0 at 1:00 in the morning.
[06:04] Because it missed the fact that you have a certain rollout procedure, you're supposed to turn off a feature flag, whatever it might be.
[06:12] So, your team needs a context engine because what it should do is understand who you are and where you work in an organization.
[06:19] So, if I say to you, I want to get off stood up, it knows where I work, it knows where my get commits are, it knows who reviews those commits,
[06:27] and it understands that my context, it can focus me, and then use that as a trigger point to find the rest of the information.
[06:34] It resolves conflicts, as mentioned, an old architecture diagram and last night's Slack convo with the CTO, which one is right?
[06:41] You need to use a bunch of techniques to discern determine that.
[06:45] Respects permissions and governance, of course.
[06:47] MCP allows us to use OAuth and other scopes and SSO, but if someone asks a question over here who's not supposed to know about secret project A, you need to make sure that doesn't leak into the response.
[06:59] And then finally, deliver the right context at the right time to the model in a token optimized way.
[07:04] We have multiple surface areas because human engineers still talk to Unblocked all the time to get information they need in Slack or otherwise, but then you want token optimized responses if you're just speaking machine to machine in order to not waste a bunch of bold classes on your token spend.
[07:19] This is how an engine works.
[07:21] I'm going to be brief on this, but basically on the left-hand side, you see all the data sources that are coming in.
[07:27] For us, we focus on engineering teams and that's who uses us, as well as the technically light teams around it, like support, sales, and otherwise.
[07:35] You ingest all that data, you get real-time data from tools like your instant management tool chain.
[07:40] It comes into the engine, where that engine is, it thinks at the bottom.
[07:44] I'll expand on that slide in a moment.
[07:46] But basically it uses these six key characteristics.
[07:49] And then on the right, you output the context to the exact workflow in the manner that it is needed.
[07:56] Those six key points, as mentioned, unified system context, you have to go across the whole thing.
[08:01] At large orgs, companies like LinkedIn scale, Workday, General Motors, whatever, they need this type of data.
[08:08] They need to understand everything that's happening.
[08:10] And Threek this morning actually talking about Fable coming out potentially later today, he mentioned that you need to actually provide a map and then let Fable discover the territory.
[08:21] The way to help confine that is making sure that these models have access to all of the context, because they will find your unknown unknowns.
[08:29] There are definitely things going on in your company that you're just unaware of, but would be really helpful for the task you're trying to do.
[08:36] That will move faster, but the targeted retrieval, you should be able to if you provide a link quickly, unfurl it, get that document back and move along.
[08:43] So, two tasks, deep research, go long, that's fine, but you also need speed when speed is required.
[08:49] Conflict resolution, we already talked about that, but one thing says do A, one thing says do B, who is right?
[08:55] Personalized relevance, who am I, where do I work, what am I working on?
[08:59] That token optimization, making sure the response is good and effective and doesn't bloat the window.
[09:05] And then permission enforcement, of course, OAuth, you shouldn't see it, you shouldn't see it.
[09:10] What we did with some tests is we actually ran the exact same prompt to the same model and one with context and one without.
[09:17] This is the wall clock time savings.
[09:19] And then 2 hours, which is great.
[09:22] And then the tokens savings.
[09:24] So, it was a sizable task, it took about 21 million tokens without and then 18, or sorry, 10.8 million tokens with it.
[09:32] This is the type of experience that you typically see when you're using a context engine, cuz the majority of those wasted search tokens where it has to grab at the beginning of every session to understand and discover things are no longer there when it's hydrated with context.
[09:45] Hydrated.
[09:47] And then, as you move forward, you get these types of outcomes.
[09:51] 50% fewer tokens, faster triage, and the answer quality is actually better because it knew what was going on inside of the business.
[09:59] Now, this next part, you'll probably want to photo.
[10:02] If you don't know, you can actually take a picture of a QR code and then later in photos tap on it and then load the link so you don't need to float here cuz I'm going to give you three QR codes.
[10:11] This first one is for the social comment network.
[10:13] I'll pop that up so you can take a photo.
[10:15] But, this is an open source tool that we've got that actually, using all deterministic programming, goes over your GitHub and understands who works on your team.
[10:22] This is my real team.
[10:23] We called Rasheem the machine cuz he ships like crazy.
[10:27] But, on the right, you can see who he commits, where he commits, who's reviewing his work.
[10:31] And then in those tabs, you can find a distilled experts graph.
[10:34] You get full coverage of what's going on in your business.
[10:36] And if you optionally add one of the API keys for either OpenAI or um Anthropic, it'll um determine what your teams are by doing some labeling for you.
[10:45] It's a really cool tool to understand where your team works and get that social network in there in order to focus the context engine if you're going to be building these tools yourself.
[10:53] The next is called the repo rules agent.
[10:56] This is a sample from our real code base.
[10:58] I'm going to pop that up anyway so you don't need to talk to the thing, but in short, what it does is discover all the places your team has written rules files,
[11:07] checks them all, and then tells you what severities you've given, what other things you've given.
[11:11] Should I just switch to this?
[11:13] It tells you what it Whoa, hey.
[11:15] It's good to meet you all.
[11:17] Basically, it will find all the rules that are inside of your repo and then tell you if you have duplicate issues or others problems and then you can grab over it as an index.
[11:26] So, that index can be called and you can dedupe and it'll help improve um your retrieval of context.
[11:32] And then finally, on Monday we delivered this workshop, which was going beyond rag and taught how to build a relational context engine from scratch.
[11:41] So, if you scan that, you'll get the full workbook.
[11:43] It has six PRs stacked that teach you how to walk through doing this.
[11:46] But in short, rag is an incredible technique and you want that.
[11:50] But the other half of the problem is what people actually ask is, "What are the open PRs that I worked on in the last week with authentication?"
[11:58] Rag cannot answer that question alone.
[12:01] You need queries.
[12:02] So, this shows you how to do a schema-less basically look up that allows the agent to discover a schema and then write queries against it deterministically in order to get that type of relational data out.
[12:14] Very useful technique.
[12:17] Use cases of a context engine, of course, do go beyond code generation.
[12:21] This is, you know, where we live a lot, a lot of our customers spend their time.
[12:25] But it's amazing to see what happens when a bunch of other people around the business start picking up these tools, customer success people solving tickets right at the time that it comes in from a customer.
[12:36] We've got sales people closing deals earlier in their quarter because they're able to just query the Unblocked context engine on the fly while in the field.
[12:44] And so many more.
[12:48] What you can also do is if you saw that curve chart earlier where I talked about the levels, we've built a fun little tool where basically an LLM will quiz you and ask you about what's going on and then it will map you to exactly where you are and then tell you some techniques
[13:00] about how to level up through that if you are looking to basically compound your capabilities and ship with AI tools at scale.
[13:07] It's readiness.unblocked.com.
[13:11] The gap is not intelligence any longer.
[13:13] It's context.
[13:14] We will continue to get incredible models like Mythos as it's been grown by Anthropic and I'm sure Soul once I'm allowed to see it.
[13:21] I will get it.
[13:22] Happy Canada Day, by the way.
[13:24] But what's happening is it's about the context you surround these models with in order for them to be effective and token efficient inside of your organization.
[13:35] So, I have a question slide, but I'm not sure I'm allowed.
[13:40] Nope.
[13:41] So, what you'll do is come meet me at booth P16.
[13:44] You can look for the coconut.
[13:46] It'll be great to hang out with all of you and get into details here if you need it.
[13:49] Thank you for your time.
[13:51] [applause]
[14:06] [music]