How Many Credentials Should Your AI Agent Have? Zero. — Jim Clark, Docker

AI Engineer · 18 min · 208 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:13I guess I'm ready to start.
00:15Uh can everyone can I I actually I can hear myself.
00:18Yeah, so you guys can hear me.
00:20Uh my name's Jim.
00:22Uh I'm a engineer at Docker.
00:25Normally when I introduce myself in the slides these days, or at least what I used to do, was I would say, "Hey, I'm Jim.
00:33I uh like spaces, not tabs.
00:36I use Neovim, not Emacs."
00:38But, that's uh all irrelevant now.
00:42So, uh I will say that I made this slide with uh I made this presentation with Claude Code.
00:49I used the Codex model.
00:51The MCPs that I that helped me write these slides were Marp and uh Mermaid.
00:57And I'm going to talk to you about MCP Gateways.
01:00But, I'm really going to be talking to you about AI safety.
01:03So, I'm going to kind of go through sandboxes, gateways, harnesses, and sort of motivate why we're talking about AI safety at Docker right now.
01:15So, like probably the last 6 months has been similar for all of you as it's been for me.
01:22Agents are doing way more than I thought they were doing uh than I thought they were going to do.
01:28And I'm no longer going to sit in front of my laptop and say, "Yes.
01:32Yes.
01:33Yes.
01:34Yes."
01:35They're doing longer running things, and I like that.
01:39And as I get more and more accustomed to them doing longer running things, I sort of decide that my metric for me having designed a problem that an agent can sink its teeth into
01:54is that I don't really need to give it a lot of supervision.
01:58But if I'm not going to supervise it, maybe I'm also a little bit worried about what it's going to get up to.
02:05So I think just to set context, this is what I think is so the sort of shape of the problem right now.
02:11We need to talk about agent harnesses, what the what the agent harnesses do for us.
02:17We need to talk about sandboxes.
02:20Like when a agent harness delivers work, what is the what is the thing it delivers work into?
02:26And then of course we've got MCPs, which which everybody knows, and we'll talk a little bit more about what we're doing there.
02:33But let's start with the agent harness.
02:36So I I mean it's it would be an oversimplification to say that an agent harness is something which just takes in context and emits tool calls.
02:48But there's also a little bit of truth to that.
02:50It's not the most complex part of it.
02:53It's a loop that takes in some context that you give it and asks you to do something on its behalf.
03:01But belaying that simplicity, we get we're all using a lot of different harnesses.
03:06And the inter the ability to why why do we pick up different harnesses all the time?
03:11Like we love the new sort of interactive interactive possibilities that they bring us.
03:17So let's just admit right from the beginning, we're going to be using a lot of harnesses.
03:21They're going to evolve pretty quickly.
03:22We're going to be swapping in different harnesses for one another.
03:27But a harness by itself is just kind of in a little bit of a dead room.
03:32Like uh hello, someone give me some context.
03:35Some somebody give me something to do.
03:38This is where this is of course where MCPs come in.
03:42MCPs allow us to do new things.
03:45They allow us to go outside of ourselves, pull in new context, pull in new tools, and and get actual work done.
03:55So, I think it's when you're trying to think about safety, I don't think it's an oversimplification to say that one of the things that you need to do is figure out what are the resources
04:06that you need to do your job.
04:08Like anytime, even for us, one of the first things you'd analyze when you want to do a new task is what do I need to get this tasks done?
04:16What information do I need?
04:18What tools do I need to actually get this done?
04:21And if you knew that the sandbox that you built had the right resources, had the right tools, and only the right those resources and tools, it would make you feel safer.
04:33It would make you feel like you limited the blast radius for what could go wrong when that agent is working.
04:39So, this is kind of where sandboxes come in.
04:43Sandbox is a place for an agent to do to do work.
04:47So, our typical sandboxes today, you're most of us are accustomed to the Yeah, we have code sandboxes.
04:55We give it We give it a bunch of code.
04:57We give it all the tools on our laptop.
04:59Maybe like when you first started using some of these agents, the sandbox was your entire laptop.
05:06As over time, we're starting to learn how to get that a bit smaller.
05:11We're trying to make the sandbox boundaries small enough that they can actually just do the task that we want to do want them to execute.
05:20That helps us feel safer about letting the agent run for longer periods of time if this if the sandboxes is a little is is is smaller.
05:30And you know, it's not that we're trying to sandbox the harness itself.
05:37The harness is kind of already sandboxed.
05:39Like it's a it's a pretty simple thing.
05:41We're trying to harness the tools and the context that are flowing into this harness.
05:47So, in order to kind of illustrate some of these concepts, I came up with two stories that I'm I'm just going to walk you through.
05:54The first one is um a newsroom analogy.
05:59So, if you've got a newspaper reporter, they go out in the world, they find cool stories, find cool information, they bring it back.
06:07Maybe someone inside of that agent agency uh look it does some fact-checking on that, make sure the information is right, and then you write a story.
06:17It's totally natural that those roles are completely separate.
06:22The actual reporter is not going to be allowed to like publish to the website or or or write write the actual article.
06:29We we break those up.
06:31That's actually how we already build complicated systems.
06:34We split them up into little pieces.
06:37So, on this slide, what you see is I've broken down three sandboxes in blue here.
06:43So, we've got a researcher, we've got a fact-checker, and we've got a reporter.
06:47So, that researcher sandbox, I think I think about that as the reporter.
06:52Now, when we define that sandbox, let it access the web.
06:56Let it access as much of the as much of the internet as we want because we're not going to give it the MCPs that it can like write to our our corporate site.
07:05It can't publish anything.
07:08We're just going to let it do we're just going to let it go out, go crazy, do research.
07:14So, our sandbox is like, "Yeah, not too many tools.
07:17I'm not going to give you too much write access, but definitely do lots of research.
07:21Just write your research out to a I don't know, to some temp directory, some private thing that is local to this agent."
07:29A fact-checker is going to need some MCPs, but it's not going to need any network.
07:34So, then let's start the fact-checker up.
07:35Let's put that in a different sandbox.
07:37Let's say no network.
07:40Yeah, and you can read the research that's been done for you, and you can fact-check it.
07:44And you can have access to our our fact database.
07:47You know, filter that out.
07:49And then finally, we get to the little blue thing on the right.
07:53And this certainly doesn't need any any access to networks.
07:57In fact, it shouldn't have any, but we do want to give it our notion MCP or our publisher MCP because it's just looking at filtered research.
08:06So, you know, look at this whole slide, draw a box around this entire slide, and that effective agent has a bunch of MCPs.
08:15Maybe it has an MCP for publishing, it has an MCP for fact-checking.
08:19At one point in time, it has access to the entire internet.
08:23But we break it up into logical sandboxes, so at each individual point, we have a don't have a dangerous concept a dangerous combination of both being able to read in crazy
08:35unfiltered context and do things with our tools.
08:40So, let's look at example number two.
08:44And I'm just going to talk about uh building a Actually, this is how I I build my coding agent right now.
08:52I tend to send the the coding agent off to do pretty long-running tasks.
08:58And let's just say that they work for about an hour.
09:03But as the as the work is happening, I also like it if it just does commits along the way.
09:11But the ratio of the amount of time that it's my agent is actually committing is almost nothing.
09:15In an In an hour, it might be it might need, say, my my my Git signing keys for like 2 minutes of that time.
09:24And when I'm committing, I need to see my my Git commit tree.
09:28I need to make that commit.
09:30I need some signing keys, but I don't need anything else.
09:33So, I would like it if the actual sandbox that an agent is running a task in is modeled after the intent of what I'm doing.
09:44So, if I'm not going to make a commit, I don't want my commit signatures, my commit signing keys inside that sandbox.
09:52And I think this this you can map across a lot of different tasks.
09:57If you know the if your agents know the intent of what you're trying to do, then use that intent to model the capabilities that you give to that.
10:07So, this is driving in in our our Docker sandbox product, this is defining a lot of how we're thinking about exposing things like an MCP gateway.
10:18So, when you build a sandbox, of course you have to put a harness in in that to do things for you.
10:24And then we're just putting a little gateway endpoint into that.
10:30And that gateway endpoint funnels all MCP traffic.
10:34So, imagine it's something like MCP an internal an an internal URL mcp.gateway.docker.internal or something like this that every single agent harness has and all traffic, all resources that you need to pull in, all tool calls move through this one
10:54this one single gateway.
10:57So, what does this buy us?
10:58Well, we end up with a control point.
11:01So, the gateway manages which tools, resources, and prompts are given to each sandbox.
11:06So, you're starting to see the beginnings of your ability to say, "Okay, I'm giving you a harness.
11:12I'm giving you this gateway endpoint.
11:14The gateway this the configuration of the sandbox controls what you can do inside of that sandbox with things like MCP."
11:23Another nice thing is suddenly all your harnesses are MCP agnostic.
11:29So, you don't have to go to codex and configure your MCPs one way and go to uh cloud code and configure your MCPs one way.
11:36You just have one you almost make the harness a parameter of the sandbox.
11:41Stick a harness in, stick the MCPs you need in, give it some network rules, Bob's your uncle.
11:49So it's it feels the MCP MCP is is a good place to manage things like credentials.
11:56In fact, I would say that a maxim that you can use here is how many credentials should be in uh a sandbox.
12:05Well, I mean the the the right answer is always zero.
12:08It just they should just never be in there.
12:12And the blast radius of an agent going doing something going going uh doing something incorrect, if there are absolutely no no credentials in there, is is reduced.
12:24So one of the things that um Docker, Entropic, Octo are all partnering on is a is a new thing called XAA, cross app application.
12:34And I think this is a really great illustration of why sandboxes and um and managing credentials in this way is important.
12:43So today in your well you're you're working at at at your companies, you have some sort of identity management, you have SSO set up, and you've got all these
12:53resource servers that that manage that that that you work with for OAuth.
12:58It could be Notion or it could be Atlassian or GitHub or Slack.
13:03They're you've already configured SSO for those.
13:07But it's not yet accessible to your agent.
13:11So with a cross app application scenario, your harness, your your your sandbox, and your gateway can define things like agent identity or who are you that this agent is behaving on behalf of?
13:28And by extension by exchanging identity claims with your identity provider, which is already there, you can get back a new thing, which is a new part of this spec that you you'll now see in the latest version of MCP,
13:42called an ID jag, an authorization grant.
13:46And with that authorization grant, we're granting access to this act to the to this actor to this agent to the same authorization servers that you've been using.
13:57There's nothing new here.
13:58We're not We're leveraging all this existing investment that these corporate internet corporate networks already have, but suddenly, without any crazy consent screens, yeah, you can do that.
14:09Yeah, you do that.
14:10Yeah, you can do that.
14:11We can centralize the administration of what an agent now does with all these existing resource servers.
14:18It's a great It's a huge simplification.
14:23So, the MCP gateways, I like to think are we're starting to talk about progressive disclosure, which is a term we learned from from skills.
14:30I like to think we're starting to use this now we're able to progressively disclose tools and resources and MCPs into these sandboxes using very, very similar principles.
14:42And of course, it's amazing because we keep contact size down.
14:46Just because you might at some point in a workflow use 50 different MCPs, doesn't mean that each sandbox has to have all 50 of those MCPs.
14:56Give your agents room to solve problems, but let individual sandboxes represent the intent of the task.
15:03Let them represent what you're actually trying to do in this task.
15:08So, I like this picture because suddenly we start to think about orchestrators as one of their jobs is to go, "Well, what am I doing?
15:16What's my task?
15:18And I need to put a task into a sandbox.
15:21So, what sandbox do I put it in?
15:23What capabilities does this need?
15:26And of course, it needs a harness.
15:28Like, what do you put in?
15:29Open code?
15:30Put in Claude?
15:31Put in Gemini?
15:32Put in Codex?
15:33It's going to need some MCPs.
15:35Take a look at the task, decide what MCPs are allowed to be used in this.
15:40Um what networking rules should you apply?
15:43Does this need access to api.github.com?
15:46Does it need access to surf the web in in in random ways?
15:49Or does it just need nothing?
15:51What resources?
15:52What work trees?
15:54We get used to thinking about modeling capabilities, building the right sandbox for the task, putting the task in, and now we're starting to see that this individual loop is safer because it has less access
16:09than the entire workflow effectively has.
16:13So, this is um this is about containers.
16:16This is about containerizing agents, which is why uh I guess this is why we're at I'm I'm at Docker.
16:23But this ties together as a kind of like role separation.
16:27What are the roles that you that your agents have?
16:30How do you map them and containerize them and and make sure that you're giving them sandboxes that express that intent.
16:37Never have any any creds anywhere in any in any harnesses.
16:41Disclose capabilities progressively into agents as they need them.
16:46The sandboxes for you are are are what actually represent intent.
16:51And if we allow some of these things to run longer, but they're more sandboxed, we feel a little bit safer about that.
17:00That's this is where safety comes from.
17:02So, we have a demo.
17:05Our our EVP of of engineering, Tushar Jain, is I got to talk upstairs today.
17:10I think it's at 4:00, and he's taking a lot of these ideas and just demo demo demo.
17:15How do you make move sandboxes to the cloud?
17:18How do you build orchestrators?
17:19He'll be really showing a lot of these things live.
17:22We also have a booth over here.
17:25Um it's a great opportunity to come over if you're interested in any of this stuff.
17:29We can show you the command line that's available today.
17:31Anyone uh just brew install sbx.
17:34That Everything I've been talking about today is this tool sbx.
17:38This is This is how we build we build containers.
17:42Um so yeah, please please uh please come over and talk to us at the booth.
17:47We'd uh we'd love to hear what you're uh what you're doing with AI safety and how we might be able to help.
17:53But uh thank you very much.
17:55[applause]
18:11[music]