WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=Uv0nAf_7tyU

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/Uv0nAf_7tyU.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:12.884 --> 00:00:15.392
Thank you very much to everyone who joined.

s2
00:00:15.392 --> 00:00:19.932
I think we'll start, since we have a full hall.

s3
00:00:19.932 --> 00:00:20.995
Hello everyone.

s4
00:00:20.995 --> 00:00:22.279
My name is Priyanka.

s5
00:00:22.279 --> 00:00:25.714
I am a technical employee at Anthropic.

s6
00:00:25.714 --> 00:00:34.536
I lead the engineering team that built Claude Managed Agents, and I'm excited to be here to chat with you and help you launch your first Managed Agent.

s7
00:00:34.536 --> 00:00:44.588
I've been at Anthropic for a little over two years, and for the last 12 months I've been focused on building out the agent infrastructure, and I hope to share our learnings with you.

s8
00:00:44.588 --> 00:00:44.926
Congratulations.

s9
00:00:44.926 --> 00:00:46.437
My name is Gabe Semack.

s10
00:00:46.437 --> 00:00:51.009
I am a technical fellow building Claude managed agents on Priyanka's team.

s11
00:00:51.009 --> 00:00:57.896
I've been at Anthropic for a little over a year and have been working on this project almost all that time.

s12
00:00:58.422 --> 00:00:58.958
OK.

s13
00:00:58.958 --> 00:01:02.825
So, here is our agenda.

s14
00:01:02.878 --> 00:01:12.697
We'll start with a history of how we built and scaled managed agents, as well as some of the pitfalls and lessons we learned along the way.

s15
00:01:12.697 --> 00:01:16.623
Then we will move on to the practical part of the seminar.

s16
00:01:16.623 --> 00:01:22.897
In the first part, we will create a very simple agent using the Claude platform console.

s17
00:01:22.897 --> 00:01:25.714
You will not write any code during this part.

s18
00:01:25.714 --> 00:01:29.748
We'll just try to create our first agent using the prompts and test it.

s19
00:01:29.748 --> 00:01:33.196
Then I'll hand it over to Gabe to walk you through the practical part.

s20
00:01:33.196 --> 00:01:42.845
We'll be writing code using Claude Coder, your favorite agent for programming, and then building a more complex financial agent, after which we'll have plenty of time at the end for questions and answers.

s21
00:01:42.845 --> 00:01:44.710
We have a two-hour slot.

s22
00:01:44.710 --> 00:01:49.011
I know it's divided into four parts, but it's essentially one long presentation.

s23
00:01:49.011 --> 00:01:51.405
I hope it doesn't take all two hours.

s24
00:01:51.405 --> 00:01:53.720
We want it to be quite interactive.

s25
00:01:53.720 --> 00:01:56.386
So if you have any questions, raise your hand.

s26
00:01:56.386 --> 00:02:04.565
We have a few people nearby who can come up to you and give you a microphone for a question, especially when we get to the practical part.

s27
00:02:04.565 --> 00:02:06.654
Okay, let's get started.

s28
00:02:06.654 --> 00:02:08.409
So, I'll start with the basics.

s29
00:02:08.409 --> 00:02:13.057
Let's start with what is an agent or what makes an agent work?

s30
00:02:13.057 --> 00:02:19.179
An agent is essentially a model in a loop with tools.

s31
00:02:19.297 --> 00:02:26.449
You will see on the left a model that makes a decision, performs the next action, and calls tools in a loop until your task is completed.

s32
00:02:26.449 --> 00:02:33.044
This is essentially the simplest definition of what an agent is.

s33
00:02:33.044 --> 00:02:35.044
Yes.

s34
00:02:44.413 --> 00:02:46.873
Link to the presentation?

s35
00:02:46.873 --> 00:02:48.408
We can try...

s36
00:02:48.408 --> 00:02:50.990
We can share it when it's finished.

s37
00:02:50.990 --> 00:02:53.217
Yes, we will share it when it is finished.

s38
00:02:53.217 --> 00:02:59.910
It will be difficult for us to provide this now, but we will be happy to answer questions and share the presentation at the end.

s39
00:02:59.910 --> 00:03:02.872
Do you have difficulty seeing slides from the back rows?

s40
00:03:02.872 --> 00:03:03.071
Good.

s41
00:03:03.071 --> 00:03:04.775
Sorry about that.

s42
00:03:04.775 --> 00:03:05.326
Um, okay.

s43
00:03:05.326 --> 00:03:09.356
The next thing I wanted to talk about is what a “harness” is .

s44
00:03:09.356 --> 00:03:18.861
Harness is usually a layer that wraps around the model and helps execute tool calls in a loop to achieve the result of your current task.

s45
00:03:18.861 --> 00:03:24.124
It makes assumptions about what the model cannot do and tries to fill in those gaps.

s46
00:03:24.124 --> 00:03:31.570
So, it works in a loop, making tool calls, managing context, and encapsulating anything the model can't do on its own.

s47
00:03:31.570 --> 00:03:38.222
A good example is our Sonnet 3.5 model, which had what we called “ contextual alerting .”

s48
00:03:38.222 --> 00:03:45.725
So whenever Sonnet 3.5 sensed that you were approaching the context limit, it tried to complete the task very quickly.

s49
00:03:45.725 --> 00:03:53.157
So when creating a wrapper around Sonnet 3.5, you added something like context reset to get around this limitation.

s50
00:03:53.157 --> 00:04:02.056
But with the next model, like Opus 3.5, there was no such limitation, and you didn't need to do such a context reset for it.

s51
00:04:02.056 --> 00:04:16.573
So when you build a wrapper, you want to encapsulate all these assumptions for different models and keep up with each new model, so building a good wrapper is very important, and that's what we tried to do with Cloud Managed Agents.

s52
00:04:16.573 --> 00:04:22.281
And lastly : you want your agent to have access to a sandbox or computer.

s53
00:04:22.281 --> 00:04:26.731
Essentially, this is where he will run the code he writes to perform the task.

s54
00:04:26.731 --> 00:04:32.293
Typically, you provide access to a cloud sandbox or container where it executes the written code.

s55
00:04:32.293 --> 00:04:37.636
It can edit files, install dependencies, etc. Cloud Code is a great example of this.

s56
00:04:37.636 --> 00:04:45.764
This is a wrapper running locally on your computer, accessing your files and executing them, updating them in the process.

s57
00:04:45.764 --> 00:04:48.674
He also writes code that is unreliable.

s58
00:04:48.674 --> 00:04:58.061
It can be buggy, so when deploying product agents you want to make sure the code they write is sandboxed and confined to the container.

s59
00:04:58.061 --> 00:05:03.247
You also need to keep your sandbox isolated and your attack radius limited.

s60
00:05:03.247 --> 00:05:13.196
Therefore, you typically want to run each task in a separate container to avoid mixing data and increasing the impact radius for your tasks.

s61
00:05:13.196 --> 00:05:19.596
Next, I'll tell you about the first architecture on which we actually built managed agents.

s62
00:05:19.596 --> 00:05:28.915
It was last December when we delivered the first architecture to our early access customers, got a ton of feedback, and then moved on to the new architecture.

s63
00:05:28.915 --> 00:05:35.041
So I'll tell you about some of the pitfalls and problems we faced in the first version.

s64
00:05:35.041 --> 00:05:39.471
In the first architecture, we placed everything in a container.

s65
00:05:39.471 --> 00:05:43.425
So, we had a session log with everything that happened during that session.

s66
00:05:43.425 --> 00:05:47.309
This session is essentially your unit of work.

s67
00:05:47.309 --> 00:05:54.064
The harness also worked inside the container , and the sandbox execution also happened inside the container .

s68
00:05:54.064 --> 00:05:59.755
The advantage was that since everything was in a container, there was no need to design service boundaries.

s69
00:05:59.755 --> 00:06:05.992
Everything was local, easy to edit, and editing files felt like system calls.

s70
00:06:05.992 --> 00:06:11.623
But it so happened that we realized: we had a " pet."

s71
00:06:11.623 --> 00:06:22.331
If you haven't heard the analogy of "pets " and "livestock" from the old infrastructure days, a " pet" is when you have a named entity that you manually care for.

s72
00:06:22.331 --> 00:06:26.079
When he is sick, you must nurse him back to life.

s73
00:06:26.079 --> 00:06:28.619
You simply cannot afford to lose it.

s74
00:06:28.619 --> 00:06:31.385
And "livestock" is something more interchangeable, disposable.

s75
00:06:31.385 --> 00:06:35.622
When one unit fails, you replace it with another and keep going .

s76
00:06:35.622 --> 00:06:38.510
A good example: imagine you have a single database layer.

s77
00:06:38.510 --> 00:06:41.403
If it falls, your product doesn't work—it's your "favorite."

s78
00:06:41.403 --> 00:06:43.795
But imagine that you have master- replica replication set up.

s79
00:06:43.795 --> 00:06:45.892
The master falls and you can pick up the replica.

s80
00:06:45.892 --> 00:06:47.510
That's what "cattle" is, right?

s81
00:06:47.510 --> 00:06:51.269
You have many instances, one fails—it's not a problem, you can restore it.

s82
00:06:51.269 --> 00:06:56.981
So, our initial setup was essentially a " pet" setup.

s83
00:06:56.981 --> 00:06:58.516
There were several problems with this.

s84
00:06:58.516 --> 00:07:01.992
Debugging was very difficult because everything was running in a container.

s85
00:07:01.992 --> 00:07:04.479
We didn't know what exactly was causing the problem.

s86
00:07:04.479 --> 00:07:13.878
A lost packet or a stopped container looked almost the same, and we had to connect via WebSocket to the container for debugging.

s87
00:07:13.878 --> 00:07:20.890
Another problem was that the investigation was difficult because these containers stored customer data.

s88
00:07:20.890 --> 00:07:24.407
So we essentially had no way to get inside for debugging.

s89
00:07:24.407 --> 00:07:28.075
This was another issue we discovered during Early Access.

s90
00:07:28.075 --> 00:07:38.316
And lastly, if a customer wanted us to connect to, say, an MCP in their VPC, that meant peering the networks or sending them our harness to run on their own.

s91
00:07:38.316 --> 00:07:45.284
So, there were a lot of problems with that architecture that we couldn't find a really good solution for.

s92
00:07:45.284 --> 00:07:53.211
So we decided that we wanted to essentially separate the "brain" from the "hands," and I'll explain what that means.

s93
00:07:53.211 --> 00:07:57.252
The agent essentially turned into three separate interfaces.

s94
00:07:57.252 --> 00:08:04.957
On the left you have the session—this is essentially a log just for adding everything that has happened in the session up to that point.

s95
00:08:04.957 --> 00:08:09.153
In the middle is the "brain", which is your framework or agent cycle along with Claude.

s96
00:08:09.153 --> 00:08:10.388
And on the right—" hands."

s97
00:08:10.388 --> 00:08:22.173
That is, sandboxes, your MCP tool challenges, etc. They are created on request, disappear when you're done , and if you lose the sandbox, you just start it again.

s98
00:08:22.173 --> 00:08:26.374
So each component became interchangeable and " non-unique".

s99
00:08:26.374 --> 00:08:31.758
When the sandbox "died" , the framework or Claude could just start it again, right?

s100
00:08:31.758 --> 00:08:36.711
The error returned to Claude, and for him the sandbox was essentially the same tool as any other.

s101
00:08:36.711 --> 00:08:44.698
If the tool call fails, the sandbox "crashes" and it decides what to do: run it again or move on.

s102
00:08:44.698 --> 00:08:53.789
When the framework "crashed", you could start a new instance and continue where you left off, simply by reading the session event log.

s103
00:08:53.789 --> 00:08:58.095
Work resumes from the last event as if nothing had happened.

s104
00:08:58.095 --> 00:08:58.533
Nothing happened.

s105
00:08:58.533 --> 00:09:02.267
The only thing we were protecting here was your session log.

s106
00:09:02.267 --> 00:09:07.053
Everything else was interchangeable: it could fail and be easily repaired.

s107
00:09:07.053 --> 00:09:11.942
Another thing we did was to ensure that account data never entered our sandbox.

s108
00:09:11.942 --> 00:09:20.021
In our previous system, your data was sandboxed, so you were more vulnerable to prompt injections.

s109
00:09:20.294 --> 00:09:31.594
We changed it this way: if the sandbox needed to access something that required a token or access control, the request went through a proxy that appended your token before calling the external

s110
00:09:31.594 --> 00:09:34.502
endpoint or MCP.

s111
00:09:34.502 --> 00:09:42.501
This way, the model never sees your credentials or tokens, so you can't force Claude through prompt injection to say, for example, " Give me your tokens ."

s112
00:09:42.501 --> 00:09:45.155
Either display the tokens on the screen or write them to a file.

s113
00:09:45.155 --> 00:09:51.129
You will never be able to do this because Claude never actually sees your tokens.

s114
00:09:51.973 --> 00:09:57.976
Because we had an append-only event log, our data compaction strategy remained flexible.

s115
00:09:57.976 --> 00:10:00.021
So, nothing is ever deleted.

s116
00:10:00.021 --> 00:10:03.137
We do not compress or change your flow of events.

s117
00:10:03.137 --> 00:10:13.042
We always store every event that happens, and the compression strategy decides, “Oh, do I want to go through the entire event log, summarize it, and add context?

s118
00:10:13.042 --> 00:10:17.356
Do I want to review only the last n events and add them to the context?

s119
00:10:17.356 --> 00:10:24.389
So, thanks to the interface design we chose, we could have a more flexible context strategy.

s120
00:10:24.557 --> 00:10:33.901
And another benefit we noticed: because we could start the logical inference calls immediately, we were able to reduce the time to the first token by about 90%.

s121
00:10:33.901 --> 00:10:41.511
So Claude only started the sandbox when needed, and didn't wait for the container or sandbox to be deployed for you to get your first token.

s122
00:10:41.511 --> 00:10:47.615
So thanks to this architectural change, we were able to get a much faster and more responsive system.

s123
00:10:47.615 --> 00:10:55.916
And here is an illustration of how one "brain" can control many " hands."

s124
00:10:55.916 --> 00:11:03.276
Your loopback agent system can now access a container in the sandbox, a customer's VPC, or an MCP server.

s125
00:11:03.276 --> 00:11:12.067
So all these "arms" are essentially interchangeable, you can connect many of them, which made our architecture much more flexible.

s126
00:11:12.067 --> 00:11:21.070
If you remember only one thing, let it be that systems contain assumptions, and assumptions can become outdated.

s127
00:11:21.070 --> 00:11:28.121
So we designed interfaces that would survive these assumptions and made the architecture more flexible.

s128
00:11:28.121 --> 00:11:41.756
This allowed us, as I said, to change the compression strategy, to completely replace the system, but the interface and architecture remain the same, and we can work with it in a more modular way.

s129
00:11:44.259 --> 00:11:49.226
Okay, now we'll move on to the practical part.

s130
00:11:49.589 --> 00:12:06.281
We've prepared free API credits for you to help you build with us today, so please scan the QR code, log in to your Cloud Platform account or create a new one, and you can get those credits to build with me and then with Gabe.

s131
00:12:06.281 --> 00:12:10.349
And I'll leave that for a moment.

s132
00:12:10.349 --> 00:12:12.349
Greetings.

s133
00:12:12.873 --> 00:12:15.558
Excuse me, what?

s134
00:12:15.558 --> 00:12:18.190
Last slide.

s135
00:12:19.490 --> 00:12:27.983
That's actually what we were talking about here , right?

s136
00:12:27.983 --> 00:12:33.600
These are kind of the three interfaces that we designed the system for, right?

s137
00:12:33.600 --> 00:12:40.262
The first question was what interfaces did we develop to keep the system relevant despite the changes?

s138
00:12:40.262 --> 00:12:55.367
So we have a session log just for appending, there's a framework that can be replaced again, and tools where you can plug in a sandbox, tool calls, or anything that the model might need access to— this makes the system more versatile and extensible.

s139
00:12:57.025 --> 00:13:01.977
Okay, back to the credits.

s140
00:13:01.977 --> 00:13:05.636
Does anyone else need to scan this before we start?

s141
00:13:05.636 --> 00:13:07.385
Okay, I still see a few phones.

s142
00:13:07.385 --> 00:13:10.073
Okay, I'll give everyone about 30 seconds.

s143
00:13:10.073 --> 00:13:12.425
Thank you for your patience.

s144
00:13:12.425 --> 00:13:18.096
Oh, are there any problems with the internet?

s145
00:13:18.096 --> 00:13:27.710
Oh no. Gabe, do we have a microphone?

s146
00:13:27.710 --> 00:13:35.291
Because I can't hear some questions.

s147
00:13:37.865 --> 00:13:42.999
Yes, the credits should arrive as soon as you sign up, hopefully within a minute or two.

s148
00:13:42.999 --> 00:13:48.906
We are now taking the microphone so that we can walk around the hall and answer these questions.

s149
00:13:48.906 --> 00:13:57.096
But I'll show you how to create a very simple agent that tells jokes.

s150
00:13:57.096 --> 00:14:06.122
I'll first talk about our core APIs and concepts, and then we'll go to platform.anthropic.com to create your first agent.

s151
00:14:06.122 --> 00:14:15.877
Our first step is to create an API for environments (V1 environments), which helps create an environment or template for your sandbox.

s152
00:14:15.877 --> 00:14:18.500
Here you will find your network settings.

s153
00:14:18.500 --> 00:14:24.298
Do you want unlimited network access or do you want to limit it to specific domains or MCPs?

s154
00:14:24.298 --> 00:14:29.829
You can specify which packages should be installed in your container or " sandbox".

s155
00:14:29.829 --> 00:14:34.482
So, this will be our first step in setting up the environment.

s156
00:14:34.482 --> 00:14:37.925
Next, you create an agent.

s157
00:14:37.925 --> 00:14:41.471
An agent is essentially an object that can be given a name.

s158
00:14:41.471 --> 00:14:44.453
In this case, we will call him a "joker."

s159
00:14:44.453 --> 00:14:48.824
We will choose the model you want to use for this agent.

s160
00:14:48.824 --> 00:14:53.382
You can use Opus, Haiku, or Sonnet—your choice.

s161
00:14:53.382 --> 00:14:57.260
After that, we will provide the agent with a system prompt.

s162
00:14:57.260 --> 00:15:03.154
In this case, we would say: "You're a comedian, find a fresh joke on a given topic and tell it well."

s163
00:15:03.154 --> 00:15:05.665
Then you can give that agent tools.

s164
00:15:05.665 --> 00:15:10.065
We will provide him with the agent toolkit, which is our basic set.

s165
00:15:10.065 --> 00:15:16.709
It's like your bash toolkit: read, write, grep, etc. Almost all agents will need this.

s166
00:15:16.709 --> 00:15:19.098
This will be your second step.

s167
00:15:19.098 --> 00:15:22.288
The third step will finally be to create a session.

s168
00:15:22.288 --> 00:15:26.719
So , the session takes the agent ID and the environment ID and creates the session.

s169
00:15:26.719 --> 00:15:29.819
This is your task, i.e., a unit of work or your assignment.

s170
00:15:29.819 --> 00:15:33.445
And when the session is initialized , it is in standby mode.

s171
00:15:33.445 --> 00:15:35.625
No work has been done yet, right?

s172
00:15:35.625 --> 00:15:37.740
Until you send the event.

s173
00:15:37.740 --> 00:15:43.130
And this is what the flow of events in your session will look like .

s174
00:15:43.130 --> 00:15:47.990
You will have events that you send, as well as events that the agent will respond to you with.

s175
00:15:47.990 --> 00:15:49.780
On the left you will see user.message.

s176
00:15:49.780 --> 00:15:52.894
You can say, "Tell me a joke about distributed systems ."

s177
00:15:52.894 --> 00:15:55.244
And then the agent will answer you with a funny joke.

s178
00:15:55.244 --> 00:15:57.883
You can use what is called a custom tool.

s179
00:15:57.883 --> 00:16:03.626
A tool essentially allows you to define the tool you want to use to handle how you respond to it.

s180
00:16:03.626 --> 00:16:14.225
And Claude will essentially add a custom tool usage event to your event stream, which you can track, process, and return the result to Claude as a custom tool result.

s181
00:16:14.225 --> 00:16:18.433
And there are a few other examples of the types of events you might see.

s182
00:16:18.433 --> 00:16:20.227
So, you will see session.

s183
00:16:20.227 --> 00:16:23.230
error if your session ever throws or encounters an error.

s184
00:16:23.230 --> 00:16:26.574
Idle status is when your session is idle and not working.

s185
00:16:26.574 --> 00:16:29.487
This usually happens when you're not actually using the sandbox.

s186
00:16:29.487 --> 00:16:36.482
And interrupt on the left is when you want to stop or interrupt the session and no longer want to continue working.

s187
00:16:36.482 --> 00:16:39.327
This is kind of our complete list of features.

s188
00:16:39.327 --> 00:16:47.292
Maybe if there's time at the end , Gabe can come back to it and dig deeper into the topic, but Gabe will work with you on his part of the workshop, looking more closely at

s189
00:16:47.292 --> 00:16:51.405
concepts like multi-agency and memory, which we also provide.

s190
00:16:51.405 --> 00:16:53.672
A. Okay.

s191
00:16:53.672 --> 00:16:59.396
Before we get to that, we'll now go to platform.claude.com.

s192
00:16:59.396 --> 00:17:03.738
I hope you can use your API credits and see them now.

s193
00:17:03.738 --> 00:17:10.786
If not, Travis walks around with a microphone, so you can reach out to him if you have any questions or need help.

s194
00:17:10.786 --> 00:17:17.643
And if you go to your console, in the agent management section you will see something called "quick start".

s195
00:17:17.643 --> 00:17:24.216
This is kind of our guide on how to create your first agent.