WTF Is the Context Layer? The Missing Infrastructure for Production Agents — Prukalpa Sankar

AI Engineer · 20 min · 265 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:12Hi everyone.
00:13My name is Prakalpa.
00:14I'm the founder of Atlan.
00:18And today I'm going to talk about this thing where context is having its moment.
00:24And so my goal today is to talk about like WTF is the context layer?
00:30Just before I start and I promise this is the last time.
00:35Um I don't know if the clicker is working.
00:57Atlan, we it's it's working.
01:00Thank you.
01:01Um the problem we solve is we say AI doesn't know your business, we fix that.
01:05We work with an incredible group of companies around the world ranging from GitLab and Zoom and Discord and Affirm to large enterprises like MasterCard and General Motors.
01:15Um And about a year ago my co-founder and I went on stage and we said uh at the dawn of the internet era Bill Gates had written this very famous blog post and it said content
01:29is king.
01:30Um and as we are the dawn of the agentic era, context will be king.
01:35Since then, it feels like 2026 is the year of context.
01:39Context graphs, anyone?
01:42Uh you know, every every two days you see some version of context popping up.
01:47And so what is going on?
01:49Um I believe the answer to this kind of is in this reality distortion field that we live in.
01:57I live here in the Bay Area.
01:59Every day or two I have conversations with people which kind of go like, "How far are we from AGI?"
02:05and we have a debate and we're like, "Well, 1 year, 3 years, so on."
02:09There is no doubt that the models are getting exponentially smarter by the day.
02:132 years ago they couldn't pass the bar.
02:15Today, if they were to take the bar, it was they're the top 1% of test scorers.
02:19On the other hand, they're not exponentially more useful by any benchmark.
02:241 out of 5, you know, AI use cases actually make it to production.
02:30You know, 56% of CEOs say that there's zero financial benefit from AI today.
02:36So, what's going on?
02:38I believe hidden in plain sight is actually how performance is measured in the human world.
02:44Cognitive intelligence doesn't really determine real-world effectiveness.
02:49In fact, only 10% of job performance variance is explained by IQ.
02:54Like, just think about it.
02:55Would you say your smartest, you know, teammate who scored the highest on the SATs is also your best teammate?
03:02Or would you say, "No, it's the person who works the most and takes the most feedback and learns the fastest?"
03:11In the real world, we care about performance and performance is outcomes that you deliver in the real world.
03:17And performance is a function of two things.
03:19It's a function of intelligence, which is cognitive horsepower.
03:22That's what the model benchmarks measure every day.
03:25But, it's also a function of context.
03:27This is what they say in the human world as learning on the job, right?
03:30Knowledge and skills and expertise that you learn over time.
03:35And in the last decade, we have compounded on one of those parameters.
03:40Intelligence has 1,000x'd in the last decade.
03:44Just in the last 6 months, we have 2x'd on that axis.
03:48On the other hand, context, the situated knowledge of your business, that's barely moved.
03:53We've moved some data to the cloud, uh but that's about it.
03:57It's otherwise logged in dashboards and Slack threads and uh the head of that analyst who might be leaving next week.
04:05Um And so, the question ahead of us then, I really believe this is the next frontier, is how do we help AI build context about our business?
04:16Um And every time I'm faced with a question about how do we help AI do this, I always like to go back and understand how did we help humans do this?
04:25Uh I'm going to take you into the life of, you know, uh uh exemplar employee, Maya.
04:32Uh let's say she's a data analyst at Mac Context Burgers, cuz I thought I was going to be creative and I'm not very creative.
04:39Um And, you know, let's say she's that analyst that everybody, you know, pings in your company.
04:46Uh right?
04:46She's the person that everybody sends a message to every morning when they're trying to solve a problem.
04:50So, let's say this morning, uh there's a franchisee owner who sends her a message and says, "Why is my drive-thru time up this week?
04:58Why is this metric up this week?"
05:00Sounds like a really simple question.
05:03Um But, it's actually a really complicated question to ask.
05:07Just to answer this one very simple question, Maya first needs to know, uh what is drive-thru time?
05:15Uh and who's asking?
05:16Is it finance or is it, you know, my ops team?
05:19And it might mean different things.
05:21Uh But, not just that.
05:23What does this week mean?
05:25Is the cut-off period Monday to Sunday?
05:27Is it Pacific time?
05:28Is it Eastern time?
05:30Uh That's knowledge.
05:31Like, that's facts.
05:32That's the map of the business.
05:34Um But, not just that.
05:36Uh there's expertise.
05:38Uh right?
05:38There's um you know, a diagnostic playbook.
05:41What What does a great analyst do?
05:43They know that, you know, quarter three is a season seasonal quarter because of weather patterns and they know to go check if the reason there's a spike is because of seasonality.
05:53They also know that the company launched a product just that previous quarter and so they know to check if that's why the root cause analysis failed.
06:03This is expertise and skills that people pick up over time as they learn on the job.
06:08And then there's norms, right?
06:10There's, you know, persona scoping, who's asking the question, how do I answer this question?
06:16And Maya, she's one of those like cool people, she nails it, she sends an answer not just with the answer but with the why and the root cause and she finds the reason for it.
06:27How did Maya learn to do this?
06:29Um she just joined the company a year ago.
06:33First, Maya, you know, has for like she joined and she got some training like all of us do, but that's not where any of us learn, right?
06:41In our companies.
06:42How do we learn?
06:43We learn because you shadow like the best teammate and then you see why they're doing something and then you learn from that and then you make a mistake.
06:52Who here has learned more from a mistake than anything else?
06:57Right?
06:58You make a mistake and then you learn.
07:01You your manager gives you feedback and you learn not to do that again.
07:05You deal with an edge case and then you learn from that.
07:08That's how all of us humans learn at work.
07:12And so then the question is how do you help build the agentic Maya?
07:18And now I want to walk you through our experiments and learnings as we've built this at Atlan.
07:23Era one and this was roughly about 18 months ago now.
07:28We started on the the track of bootstrapping agents.
07:34And the way we went about it was and we started this with our customer experience team and we did this jobs to be done analysis map, right?
07:42And so we said, "Hey, if you are someone on our customer experience team, what are all the things that you do on a day-to-day basis?"
07:49And then we made some hypothesis.
07:51We We said, you know, for example, one part of the job is documentation and meeting prep.
07:56Uh we said, "Well, AI could probably do that job pretty well."
08:00Uh and so we built the scaling factor.
08:02So, on the other hand, relationship management is something that our customer experience team does.
08:06And we said, "Hmm, that doesn't sound like something AI is going to be able to do anytime soon."
08:10And so we built the scaling factor.
08:13[snorts]
08:13And then we basically started bootstrapping these individual agents that were like built for that specific topic.
08:23Uh our team got creative, so we had Hermione, who is our health intelligence lead, and then we had, you know, MoneyPenny, who was our financial risk analyst, and we just made that particular agent
08:34really good at doing that one thing.
08:38Um and that worked for some time.
08:41Um but then we realized there were some challenges with this approach.
08:46The first, context engineering.
08:48Uh we got to the point by middle of last year where building an agent was really easy.
08:52Took like 5 minutes.
08:54Uh but giving it the business context that it took to actually get it to be accurate took forever.
09:00Uh quality of the agent often dependent uh on the quality of context engineering, and that led to a lot of weird lost trust cases with our stakeholders.
09:10[snorts]
09:11Um then, as we started taking this into production, we started seeing that these agents basically were kind of like living on their own island.
09:20Uh now imagine, for example, if you're in a human team, and your marketing changes positioning on your, you know, and then they come to the town hall and they tell you that they changed positioning.
09:30And so then, you know, the SDR on your team or your sales development rep, they know that they should use is new positioning.
09:36This is like the infrastructure that we've built for humans inside our organizations.
09:40Agents didn't have them.
09:42So, our marketing team had these agents and they started making changes to that and then our SDR agent on our website was still pitching the old version.
09:51Uh we had no idea how any of these things were even connected.
09:54So, we didn't even know how to like run this as as a team of agents.
09:58Uh when an agent gets something wrong, this is hard.
10:02Uh it was really hard to like trace back what happened.
10:04Was it the model?
10:06The agent?
10:06Was it the context?
10:07Like where How do we even go back and fix this?
10:10Um and over time we started dealing with context sprawl.
10:15Uh we had the The hard part about this was agents all had their own memory systems to a certain extent.
10:23So, they were learning They were all learning separately and they were learning differently.
10:27Uh it became very, very difficult very quickly to say, "Okay, what does the single version of truth here look like?"
10:34Um and then over time we actually went through In the last 12 months we've gone through cycles of at the agent tech layer.
10:41About 12 months ago we were using one of these no-code type builders uh called Relevance.
10:46We went from there into Google ADK, then we tried Glean.
10:50Uh start of this year we moved to Claude Code.
10:53Now we are kind of like 50/50 Claude and Codex.
10:56Um and every single time as these changes happened, uh our context got trapped in each of these individual systems.
11:04Um So, starting this year as general purpose agents started to become a thing, we said, "What if there was a different approach with general purpose agents?"
11:15Uh again, going back to the human world, well, Maya, she's not an individual star.
11:20She's part of a team, right?
11:21And you know, you talk about these dream teams like Maya and someone who runs customer support and someone who launches ads.
11:28These people work really well together.
11:30And often these dream teams are built on shared context, right?
11:35They have a shared language.
11:37They have a shared picture of what's true today.
11:40They have shared playbooks.
11:43They have shared norms, who's allowed to make what decision.
11:46And then they learn together.
11:48I think this is the most important part of it.
11:49They have compounding learning loops of what good looks like.
11:53And they have shared memory that, you know, oh, we launched this thing last quarter and it like was terrible and we're not going to make that mistake again, right?
12:01And so we said, is there a way to bring that into the way we think about AI in our companies?
12:07And so the mental model we started working on was we said, okay, we have these teams of humans and they're across the board and can these people essentially start building domain skills.
12:19So each of them is responsible for a certain set of skills.
12:23All of this goes into this common one place, which is this one company brain of sorts, right?
12:30I like to think of this as the context layer.
12:32And then this has a bunch of retrieval mechanisms, which then talks to the general purpose agent across the ecosystem.
12:41So then we started the experiment.
12:44This is some version of what our marketing team ended up building.
12:47So you'll see on the left, those are all the systems that our marketing team uses.
12:51So data systems, our social and community platforms, our ad platforms, our analytics platforms.
12:58And then you'll see this agent block.
13:02We built this very specifically for having openness.
13:07So
13:07[snorts]
13:08we had Cloud Code and Co-work.
13:10We also had our own claw that we deployed, which has, you know, essentially talks in our Slack channels.
13:17And then we used some external products like Qualified and Artisan.
13:21[snorts]
13:22In the middle is kind of this context layer that our team started building.
13:26So think of it as our best SEO person was building the SEO skill.
13:32Uh our best competitive intel person was building the best competitive intel skill.
13:36And that kind of became this common repo that we were building into and pulling out from.
13:42[snorts]
13:43This sort of became our living brain.
13:46Over time, we realized there were some things that we needed in this brain, right?
13:49Uh we realized we needed a data graph.
13:51Like if For example, our autonomous ads agent we realized it needs to do analysis on a daily basis.
13:57So, like which table should I go pull from?
14:00Uh we needed a library of skills.
14:02We also needed some other things, semantics, metrics, what is ARR, how do you measure that, uh what is a qualified lead in our company, uh and org structure, entities, things like that.
14:16Over the last 6 months, we ended up creating about 300 skills and 40 agents in this team, uh which has been incredible.
14:24Uh but then, with this approach, too, we realized that there were some challenges.
14:28We realized that context kind of needs to be managed like code.
14:34Um so, some challenges.
14:37Let's pick skills.
14:38Uh dependency management became really complicated.
14:41So, for example, we have this competitive intelligence skill, and it learns from the market on what's changing in the market, and it improves.
14:49Um it feeds our category positioning skill, which then feeds our sales battle card skill.
14:57Uh now, each of these skills is learning and evolving, uh but every time they learn and evolve, it breaks something downstream.
15:04Uh and these skills very quickly start getting outdated and start drifting.
15:08Uh who owns skill quality became another thing.
15:10Like who eventually owns the quality of this?
15:14Security and governance was a nightmare.
15:16Uh we had secrets hardcoded in .env files.
15:19Uh it was People were downloading these public skill repos, this the whole thing was like a nightmare.
15:25And then I talked about context portability across all these multi-agent systems.
15:31I started this talk by saying WTF is a context layer.
15:35These are the problems that a context layer is meant to solve.
15:39The question I like to ask is what does the GitHub for context look like?
15:45Few thoughts.
15:47Company context needs life cycle management and collaboration and versioning just like code does.
15:56You know, there's questions like what's local context, what's global context, how do I keep this updated, so on.
16:02Some thoughts in this, can skills have a profile just like code does?
16:09Can that have a self-learning loop that's baked into it?
16:13What does quality management look like?
16:15Can you have security and postures posture management associated with that?
16:20That's really like the first step.
16:23I see this as like having something that has built-in versioning and quality and dependency management.
16:28So, you should be able to say, "Hey, this thing impacts all these other things.
16:31This is the approver, this is the maintainer, these are the contributors.
16:35How do you build that kind of human plus AI workspaces that these that these skills are managed via?"
16:41[snorts]
16:43Second thing, every AI interaction creates more context and harnessing this is gold.
16:50There's been I know a lot of talks about self-improving loops.
16:53We have found that with traces deploying a specific harness that actually is specialized in being able to go and reverse construct from that.
17:02So, think of it as AI that's reading through all your traces and almost brings it back to your maintainer loop and says, "Approve, reject.
17:10Approve, reject.
17:11Improve this over time."
17:13That's the compounding learning loop.
17:16And the third, often a lot of people ask me this question, which is like, how do I start?
17:21Because my business is like really disparate, and I have all these like 60 systems, and how do I even start?
17:26One of the biggest learnings we've had is context is hidden in these in business systems.
17:31Uh and across this context quality can really compound.
17:36So, for example, if you're able to connect your Salesforce and your HubSpot to your data warehouse, to your application layer, and then you're able to reverse construct how these things are actually connected one to another,
17:47context today gets lost in every one of those hops.
17:50But if you can reverse construct that, and then deploy AI on top of it, we've seen incredible accuracy in being able to reverse construct the first version of your company brain.
18:02[snorts]
18:05So, I'll end with this.
18:07Uh the way I think about a context layer is it's a system that turns knowledge and expertise and norms that we talked about, that Maya knows, into a machine usable context
18:18for AI systems.
18:20Uh At a very high level, the way I like to think of it is it looks like this.
18:27Uh It continually is mining context from your business systems.
18:32It's feeding this in to that one company brain.
18:35It's harnessing this in skills and context development life cycles as your teams go and deploy these agents.
18:42And then it has a bunch of ways you can retrieve it.
18:45So, MCP, SQL, vector retrieval, hybrid assembly, all these different ways that you retrieve it, and pull back from traces and build this compounding learning loop.
18:55[snorts]
18:58Today, we're largely building agents by hardcoding context.
19:07The scale of this problem I truly believe is underhyped, because with scale this can become really unsustainable.
19:13Uh And a little dangerous.
19:15Like all of us know this this old joke, which is if you ask sales and finance the revenue number, you're going to get two different numbers.
19:23We're fast approaching a moment of starting to deploy autonomous systems where the same thing is starting to happen.
19:31So, I'll end with one last thing.
19:32I started this presentation by saying context is king.
19:37I'd like to end it by saying context is also IP.
19:42Something I think a lot about is in a world where you and your competitor have access to the same models and the same intelligence, what differentiates a company?
19:52What differentiates a customer support agent at American Express versus Amazon?
19:59That's how you do business.
20:01That's what makes your company special.
20:04Context is how we take and encode our culture and our norms into something that we will be proud of as we build autonomous frontier firms.
20:15Um And that's all I had.
20:19Um you can find me at Prokalpa on Twitter um or write to me.
20:24We are actively working with folks on the frontier on going and shipping and building company brains.
20:30Um so, if you'd like to talk to us, feel free to reach out.
20:33Thank you.
20:35[applause]
20:49[music]