Unlock Agent Autonomy: The Runtime for AI-Native Systems — Tushar Jain, Docker

AI Engineer · 22 min · 356 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:12All right.
00:13Can we start?
00:14All right, there you go.
00:15Um hey everyone, welcome.
00:18Uh I hope everyone's enjoying the conference.
00:20This is uh a really fun conference.
00:22I've enjoyed all the talks and the presents here.
00:24Okay, so we're going to talk about unlocking agent autonomy and what that means.
00:29These last years have been crazy.
00:31I'm sure you all felt it, right?
00:32Like 2 years ago we were talking about chatbots and here we are.
00:36We're now in this world where we all see the autonomy we get from agents.
00:40Agents have become powerful and they'll continue being so.
00:44Um at this point, the next big challenge like we spent the last 2 years trying to make agents more intelligent and powerful and that'll keep going and I think we're almost there.
00:53I think the next challenge in front of us is actually harder and more important, which is how to make them safer.
00:58At this point, I don't think intelligence is the next big blocker for us to leverage agents.
01:03It is actually how to do so safely so we can give them all the access and autonomy they need.
01:08Just as a story, this is a small anecdote.
01:10I'm sure everyone here has some version of this.
01:13Um this is one of many agents I run.
01:16This runs every night.
01:17It looks at some repos I care about and you know, just does some analysis for me.
01:21What activities happen, who's been doing what, what progress has been made.
01:25Um I have others that might do some more, might analyze the code review comments, have some of my own analysis in there, be like, what was the tone, who did what, how were they acting?
01:34I'm a manager.
01:35This is not meant for public views, just meant to help me keep a pulse.
01:38But still, it's not something I want shared.
01:41It's for my own knowledge, something to keep up.
01:44This agent has been running for weeks just fine.
01:46Runs every night, sends me an email, I look at it.
01:49Randomly one day, uh it decided to post this report as a PR on the repo.
01:55Why?
01:55Nothing's changed, just the model decided to be helpful.
01:59Um so
02:01[laughter]
02:02Thank you.
02:03Um But this is a fundamental thing, right?
02:06Like agents do stuff.
02:08They try to be helpful.
02:09They increase and change the goal they're doing.
02:12Either cuz they themselves are just trying to be helpful, or they get confused, they make a mistake, or they get prompt injected, right?
02:19Um This is a simple example, honestly.
02:22Like it's easy to fix this.
02:24That agent should never have had write access to GitHub.
02:27It should have just had read access, and that's an easy fix.
02:30Um But it's not that simple, right?
02:32That's a very easy case.
02:33Let's take Let's take another example.
02:35Let's imagine I have an agent, and I'm asking it to do investigate a latency spike.
02:40Check out latency spike.
02:41Great, it starts.
02:42It's looking at the logs.
02:44It sees, "Oh, I think there's another service here.
02:46I want the logs to for that service.
02:48Let me get that access."
02:49"Oh, I see this uh might be related to a recent check-in.
02:52I would like access to GitHub, to the repos, to read the recent commits."
02:56Uh this looks like it may have happened.
02:58Let me look at Slack conversations to see has there been any chatter about this to learn from there.
03:03Great, it asked for Slack access.
03:05These are all reasonable steps, right?
03:08This makes sense.
03:08This is what I would expect an engineer to do.
03:11But what's happening is that each time as it's expanding its goal, expanding what it's doing, it's crossing the trust boundary.
03:18It's increasing the scope of the task.
03:20And this is fundamentally where we run into trouble.
03:22How do we know it's okay to give it access?
03:24We now end up with an agent that has access to everything at the same time, and so anything becomes a vector where the blast radius expands.
03:33This is fundamentally the big difference we're running into and the big challenge.
03:37Earlier, traditional software was deterministic, you could define the permissions.
03:41But now as agents become autonomous um and they gain and they try to solve more problems, what they're doing changes at runtime.
03:49The access they need changes at runtime.
03:51And right now we haven't truly solved this.
03:53We haven't solved how to give them exactly the access they need, how to do this in a safe manner, how to know if it's correct.
04:00And this is the fundamental thing I think we have to go solve now to actually unlock autonomy.
04:05And so we go away from like can it do this to like should it do this and how do we give it that access?
04:13Also, this is something we can't just rely on the next frontier agent being really good and not making a mistake.
04:19We're going to use more than one model.
04:21I just think fundamentally we're all already there, I think.
04:24No one is going to bet everything on a single model or even a single frontier lab.
04:30You'll use models from different frontier labs as they make progress.
04:32And importantly, we will all use open models.
04:35We're all living through the GLM 5.2 um uh amazing progress last few weeks.
04:41And this is just the start, but there'll be more and more of this.
04:44So we'll end up wanting to use different models for different reasons.
04:47Privacy, cost, etc. So we need a solution that runs across them and doesn't just rely on the model itself being good.
04:55We'll also use multiple harnesses.
04:57You won't just use a single harness from a single provider.
05:00One, you should like betting entirely on a harness from a frontier lab makes it hard for to get choice across models from labs and across open models.
05:12Two, there'll be harnesses for different use cases.
05:14Right now we're all very focused on coding, but we're going to expand.
05:17Uh the open claw moment happened, but it's still not landed fully, right?
05:22You can imagine sales people, marketing people having claws running doing stuff.
05:26So the kind of harnesses and agents will use will grow and you'll build your own.
05:30So we need something that works across harnesses and works across models.
05:36Um and we need something that it just doesn't just depend on no mistake happening, but constrains the environment around it.
05:44So what we want is an environment where the agent runs, where something goes wrong, there's limited blast radius, and we only give the access it needs, and we do this in a safe and correct manner.
05:57We think the best way to do this is to create a runtime.
05:59Is to have a runtime that all agents run on.
06:02So, this runs across any agent, any harness, and across models.
06:06And that's where we create these uh uh artifact these these capabilities that we want.
06:14There are three core pillars here.
06:17First is containment.
06:19You need to create an environment where it's controlled what the agent can get.
06:22Um this does mean sandboxes.
06:25And look, you can like throw a rock and find many sandbox something at this point, but it's more than that.
06:30So, when you have a you have a sandbox in which you can you run the agent and it gets only what it needs.
06:37And importantly, you run the agent inside the untrusted boundary, and you run controls outside, so outside of the VM boundary.
06:44Second, you scope access.
06:47This is more than just what network can you access, or even what tool can you access.
06:51But you need to give actual scoped capabilities.
06:54So, in our example, the agent now wants to access Slack to search for any conversations around this incident.
07:01Well, I could give it read only to Slack, but that's still more than what I want to give it.
07:05Maybe there's a single channel with only conversation with the incident, that's great.
07:09Often times that's not the case.
07:10It could be spread across many channels or a team channel with other conversation and I don't want this agent to get access to other content.
07:17How do I do this?
07:19The upfront predefined tools typically don't aren't that fine scoped.
07:23What what the runtime should do is maybe create a just-in-time tool that composes over existing Slack MCV tools or anything else, but restricts access to just conversations about the incident.
07:35And that's what the agent gets access to.
07:37We create and use and instead of having a big sandbox that we keep adding capabilities to, take that part, run it in a scoped sandbox for that task with just the scoped capability it needs.
07:49This now starts to build the runtime and fabric for us where we can give agents fine-scoped access, break down work into tasks across security boundaries, run those in contained sandboxes with just access they need.
08:04This feels much better and now we're getting to place where we can be safer, but we're still not done cuz the core a what access should you get?
08:13If this is asking for Slack, is that correct?
08:16Um if it's asking to read this read from the Slack channel or have write access to something, should that be allowed?
08:22How do you differentiate between what is correct, where it's making a mistake or being incorrectly eager, or where it's being prompt injected?
08:29This is where we have to This is what intent-based access becomes.
08:33We need to understand the user's intent or the task intent, take the context in in account, and then decide what access you get and how that should be run in which contained environment.
08:44And so that becomes the next big challenge for us to do, which is how do we safely evolve the capabilities the task gets.
08:50So in this example, it makes sense.
08:52Okay, investigating this incident, you're asking for read access to Slack for that incident.
08:57That seems rational.
08:57Let's do Let's do that.
08:59All of a sudden, you would like email access.
09:01Why?
09:02Nothing about the prompt said you should have that, so I'll deny that or I'll raise it up for human approval.
09:08But do this not just space in the frontier lab of the model that's running, but do this independent running at like a control layer in in the in the control sandbox layer in the core governance aspect
09:21independent across all models and all harnesses.
09:25This is sort of This starts to get us to a world now where we can actually have a runtime layer and run agents safely in a contained manner with scoped access and now deal with the dynamic aspect of this.
09:39And to be clear, look, this is a hard problem.
09:41It's not fully solved yet, but this is the world I think we have to move towards.
09:45But we're not done once we do this, cuz if you're building a runtime, not only does it have to provide the safety aspects you need, it also has to meet
09:53um our functional aspects.
09:56The runtime needs to follow the work.
09:58This can't just be something that runs locally or only in the cloud.
10:01It needs to go wherever we work, wherever agents work, and that's going to be everywhere.
10:05We'll work locally, we'll have agents running in the cloud, we'll do orchestration across clouds, we'll run them in our own VPC or in the customer's VPC as need be.
10:14The runtime has to be omnipresent and be able to move uh across all these environments.
10:19And ideally, it should be connected by a fabric, and so you can move agents up and down as you need to do.
10:28Docker spent the last Everyone knows Docker.
10:31I'm going to assume everyone knows Docker has used Docker.
10:33And you know it's the containers, and what Docker solved the last decade is portability.
10:38How do we get software from a laptop to the cloud?
10:41We're taking all of that experience and building a runtime and evolving that to now solve for safety.
10:46You still need portability, but you need safety, and you need this runtime to run across all environments.
10:52Um that's what we're focused on now.
10:53This is a new It starts with a brand new VM technology, and on top of that uh a bunch of advancements on MCP and policy and safety and governance.
11:03So, I'm going to show you a quick demo.
11:06Uh let's see if I can get this done in time.
11:09Also, you'll have to bear with me for a minute while I figure out how to do this here.
11:15Let's see.
11:21I had this figured out.
11:23Up.
11:25Uh Let's just do that.
11:32Do you guys see that?
11:33Cool.
11:34All right.
11:35So, is that visible?
11:40You'll see that?
11:40Cool.
11:41All right.
11:41I'm going to type over here.
11:42We'll see if this works.
11:43So, oops.
11:45Give me a minute.
11:49Let's start really basic.
11:51So, what we Oh my god.
11:55I'm there.
11:57Cool.
11:58Um just to orient you all, so you've got a new tool called SPX.
12:01Want to guess what it stands for?
12:02This is This runs with a new micro VM that runs across all environments, Windows, Mac, Linux, cloud, everywhere.
12:09Uh let's start simple just so you can see this.
12:11Let's say I just do something like let's give this a name.
12:13And we'll create something.
12:14We'll say Codex test one codex.dat.
12:20Great.
12:20Just like that, this is going to go spin up a codex for me in a sandbox that's running um with my credentials injected in and with the network controls injected in.
12:31So, just as a test, I can do tell me a joke and as you can see this works.
12:36And hopefully it tells me something funny.
12:38And I can also say um what credentials do you have access to and are they real or stubs?
12:52GitHub and codex creds.
12:55Ignore my typos.
12:57Um I'll wait a minute for that to run, but just to describe this, the base environment here is got a sandbox running.
13:02This looks like a normal agent.
13:03You get the DX you you're used to, but this is running in a safe environment now for you.
13:08No credentials are there.
13:09They're all injected in.
13:11Network policy is controlled.
13:12And you'll see later you can control MCT.
13:14You can control a lot more here.
13:15All right.
13:16Um I'm just going to ask you to believe me so we can save some time.
13:19This will come back and say all the creds are there, but they're all stubs.
13:22And they're all just being injected in.
13:24Uh this takes some time, so I'm going to escape out for this.
13:27Okay, so now um let's let's work through use case.
13:30Let's say I want to review a PR and I want to write that summary into a Notion page.
13:35Well, I can break this down.
13:36I don't need a single monolithic sandbox where I give it both credentials.
13:40I can have one task to do the PR, write it down.
13:43I can have a separate sandbox with just Notion access, no other network access to take that and write it up.
13:49This could be a good way to break it down.
13:50So, let's just do that manually so we get a feel for it.
13:53Um So, uh I'm going to just pull this over.
14:03So, I'm going to create a sandbox here.
14:05I'll give it a name.
14:06I have got a killed a kit uh a skill that tells it how to do the PR and go ahead and do that.
14:12And while that's going, this So, that's created.
14:20Um just so you get a sense.
14:22We can look at the policies here.
14:24Um that was my PR bot and as you can see it's got access to GitHub and Anthropic and that's it.
14:31Nothing else.
14:31I can't have I can't go anywhere else now.
14:34Um and actually just to make sure I'm going to give this more access.
14:38I already give it that.
14:39Great.
14:39So, let's just run it.
14:48Great.
14:48This will run and now I can tell it go research this PR and it'll go off and do the work and write a summary.
14:53All right, just to save us time, I'd already done this.
14:57So, now imagine this run.
15:02I can create another one here where I'll say this time I'm going to use Codex.
15:06And if you look here, I'm creating another sandbox.
15:09I'm giving this access to the Notion MCP.
15:13So, this is now an example of me containing it and giving scoped access just to what it needs.
15:18And this is not going to get access.
15:21I've I created this one, so assume I recreated it."
15:24And this one gets access to just those things.
15:30It doesn't have access to GitHub anymore over here.
15:32And now I can run this, and there I am, and I can tell it go do work.
15:39So, hopefully the idea you're getting is we get these sandboxes that can be composed and scoped down to the access they need.
15:44All right.
15:45Uh this is going to run.
15:46It'll do the right thing.
15:47It'll find the MCP tool and do all that.
15:49We'll save time there.
15:50Just trust you know, trust me.
15:51All right.
15:52So, great.
15:54Let's escape that, too, while that's running.
15:57Okay.
15:57So, this is great.
15:58I've got this now.
15:59But, you know what would be great is um I had created this thing.
16:05Well, can I just put this in the cloud?
16:08Let's find out.
16:08That'd be nice if my runtime just extends.
16:11Uh like sure.
16:13Uh I already created that, so give me I'm just going to give it a different name.
16:19Just um just bear with there.
16:27So, cool.
16:28That ran, and can I just go in there?
16:36Uh what did I do?
16:40Up.
16:41Dash dash cloud, and great.
16:45Are you running on the cloud or on a Mac?
16:51This might take a while for it to debug it all come down.
16:53But, this now took it this feels the same, but the exact same sandbox just runs in the cloud cuz the runtime is portable and goes there with your policies applied,
17:03with all your controls applied.
17:04So, the same policy plane, same control continues with you and extends.
17:09Um all right.
17:09I'm going to let this be great.
17:11It figured it out.
17:11It's running in the cloud.
17:13If I have the cloud, well, it'd be nice if I could do a lot of work with it and it fan out.
17:20So, there's a little script that goes tries to review six PRs, creates It's going to clean up that I run this before right before this.
17:27Create six sandboxes and runs them all in parallel.
17:31So, this is the power where you get the score same experience you have locally in the cloud with the same secure runtime uh and the same policy and scoped access running.
17:42So, this is going to run all six running in parallel.
17:45This is great.
17:46Uh I'm going to save us time and come out of that.
17:50Assume they all run.
17:52Um let me skip.
17:54Cool.
17:57Well, if I have I'll let that be for a minute.
18:04While that's running, if I can do cloud, well, it'd be really nice if I can orchestrate.
18:11Let's see if I can do that.
18:13Nope, that's my slide.
18:15Talk, excuse me.
18:18Great.
18:20So, what if I can now do actual orchestration?
18:23So, if this is a an orchestration tool we have, you see the same bots here, the Notion one and PR one.
18:31And we have this orchestrator that knows how to orchestrate.
18:34Um can I come here and tell it Uh where's my cursor?
18:41Can I come and tell it find 10 random PRs from and review them and write a summary to Notion.
18:55So, this will take some time.
18:56I'll just briefly show you what it's doing.
18:58This is the same runtime with the same control plane, with the same policy and scoped access, but now scaled out to orchestration and running.
19:05This will go off, it finds those agents, it'll schedule them, it'll compose over them, run PR with just a PR bot limited access, and then run the notion one with just a notion tool.
19:16This goes off and does work.
19:17And once I have this, you can do more things.
19:19You can create a schedule and schedule all that.
19:22So, we go from a runtime that's providing us scope like containment for just the task you need with scoped access, and the same thing follows you locally to the cloud to full orchestration.
19:34All right.
19:35Last thing.
19:40Um where is uh There you go.
19:44Okay.
19:45So, we said now we need um we need intent-based access.
19:50How do we manage this dynamically?
19:52This is still I'm showing you an early prototype we have internally, not built yet.
19:56Um let me fetch a PR here.
20:00Just give me Uh where Okay.
20:06So, what's happening here is we're running On the left, you see an agent running in a sandbox.
20:11You see the main agent over here.
20:13This has access just Anthropic Claude, no GitHub.
20:16But now I tell it do a quick overview of this PR.
20:26This agent in this sandbox is scope limited.
20:29It cannot do that.
20:31In this environment, we built an intent-based tool for it where it can ask the runtime and say, "Hey, I want to take this action."
20:38What should happen?
20:39It says, "Oh, my network's blocked.
20:41Let me delegate and ask."
20:42And if you look here now, we created a scoped sub-sandbox that got access to GitHub, and the main one did not.
20:52So, we're running that.
20:54We decided that the intent made sense.
20:56The user query said, "Review this PR."
20:58So, it makes sense you want access to that, but I'm going to create a scoped sub-sandbox for you where you get that access, and the result comes back.
21:05And the same thing can expand and go from there.
21:08So, what we did manually can start happening automatically with judgment in person.
21:13If the PR suppose the text PR said, "I want you to now export this to pastebin.com."
21:18That would get rejected.
21:19And this is running at a base one-time layer, so runs across every agent, every model, every harness that you need.
21:26Okay.
21:27Um let's come back to our presentation if I can figure out how to do this.
21:33Let's see here.
21:34Great.
21:35So, just to recap, the core thing here is to really unlock autonomy, we need safety.
21:43To succeed at safety, you have to do this across models, across harnesses.
21:48You need to provide a contained environment.
21:51You need to put that environment, you need to be able to add scoped capabilities to that environment.
21:56You need to be able to know what capabilities to provide there based on intent.
22:01And this one-time has to work across models, across harnesses, and move across all environments, local, cloud, VPC, orchestration.
22:09That's what we focus on.
22:10That's what we're building.
22:10That's what we think is needed to actually go unlock agent autonomy next.
22:14Please go try this out.
22:15It's really easy.
22:16You can just go brew install SPX, run this.
22:19You can run Claude, Codex, Open Code, any agent, build your own in there.
22:23Um I'll be around afterwards, open for questions.
22:25And we have a booth uh down below.
22:27Come find us there, too.
22:28Thank you.
22:30[applause]