WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=FGftGOzfohc NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/FGftGOzfohc.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:01.309 --> 00:00:03.309 [music] s2 00:00:12.559 --> 00:00:14.160 Thank you for the presentation. s3 00:00:14.160 --> 00:00:22.480 Out of curiosity for the design of the session in um first iteration and second iteration what has changed? s4 00:00:22.480 --> 00:00:23.119 Good question. s5 00:00:23.119 --> 00:00:31.679 Gabe, do you want to take that question on what changed within uh the storage of the session between the first and the second iteration? s6 00:00:31.920 --> 00:00:34.160 Um, sure. s7 00:00:34.160 --> 00:00:48.079 The the the main thing that changed was that we wanted to abstract away as many things and comp compartmentalize them into different components um to make it so that we could extend the storage mechanisms for future events that we wanted to store s8 00:00:48.079 --> 00:00:53.280 uh without tying them you know so tightly coupled to the mechanics of the session work. s9 00:00:53.280 --> 00:01:04.640 Um, so we basically apply the same principle of building and virtualizing interfaces all the way down uh not only across service boundaries but also at the event layer. s10 00:01:04.640 --> 00:01:05.600 All right, thank you. s11 00:01:05.600 --> 00:01:18.880 Um, so it sounds like maybe everyone might not be able to follow along, but hopefully um we can find a way to share out these slides and you can try it out at home or if you have your phone, maybe you can try to pair it with your hotspot and see if you can s12 00:01:18.880 --> 00:01:19.840 follow along. s13 00:01:19.840 --> 00:01:20.479 Um, all right. s14 00:01:20.479 --> 00:01:21.920 So let let's try this one more time. s15 00:01:21.920 --> 00:01:33.920 And we're going to say, build me an agent that tells me jokes every time given a topic. s16 00:01:33.920 --> 00:01:36.880 And then you just hit enter. s17 00:01:39.119 --> 00:01:44.159 And you can see Claude is coming up with a system prompt. s18 00:01:44.159 --> 00:01:47.119 It came up with a name, a description, chose a model. s19 00:01:47.119 --> 00:01:50.000 You can like go change this model if you want to. s20 00:01:50.000 --> 00:01:53.680 and um give the agent access to the agent tool set. s21 00:01:53.680 --> 00:01:57.360 And then you'll hit uh create agent. s22 00:01:57.360 --> 00:02:01.439 That should basically call the v1 agents endpoint to create the agent. s23 00:02:01.439 --> 00:02:03.200 Okay, that that got created. s24 00:02:03.200 --> 00:02:09.360 Next, we're going to configure the environment uh which is the template for the sandbox. s25 00:02:12.080 --> 00:02:22.319 Uh we're just going to create a new one and then we'll just give it unrestricted access for this one. s26 00:02:22.319 --> 00:02:31.360 But you can obviously when you're building a production agent you you want to tighten the access control which domains it can uh reach out to or specific MCP servers. s27 00:02:31.360 --> 00:02:33.920 So we'll do unrestricted. s28 00:02:35.440 --> 00:02:43.519 And then here you can see you can like choose packages you want to install um etc. And then next we'll just go and start the session. s29 00:02:44.959 --> 00:02:45.440 All right. s30 00:02:45.440 --> 00:02:48.879 Now we're going to go test run our session. s31 00:02:48.879 --> 00:02:49.680 Uh right. s32 00:02:49.680 --> 00:02:52.720 Tell me jokes about cats. s33 00:02:54.879 --> 00:03:06.319 And you'll see the model started thinking and it came up with a cat joke. s34 00:03:06.319 --> 00:03:06.640 Yeah. s35 00:03:06.640 --> 00:03:10.560 That that was like a that that's how easy it is for you to go build your agent. s36 00:03:10.560 --> 00:03:13.120 Now this uh agent is up and running in the cloud. s37 00:03:13.120 --> 00:03:17.920 All you have to do is start a session with the agent, send events, get back events. s38 00:03:17.920 --> 00:03:19.680 Everything else is managed by us. s39 00:03:19.680 --> 00:03:27.040 So we're managing all the infrastructure, the hosting, the harness, the sandbox management, the scaling, the storage of your event log. s40 00:03:27.040 --> 00:03:30.799 Um and then you can focus on building your product around this. s41 00:03:30.799 --> 00:03:31.680 Um thank you. s42 00:03:31.680 --> 00:03:39.840 I'll now hand it over to Gabe to do the more hands-on bit of the workshop and hopefully hopefully we have better luck with Wi-Fi. s43 00:03:39.840 --> 00:03:42.000 Thank you. s44 00:03:42.702 --> 00:03:44.702 [applause] s45 00:03:47.360 --> 00:03:48.239 Can you hear me? s46 00:03:48.239 --> 00:03:48.560 Great. s47 00:03:48.560 --> 00:03:51.120 I see there's a bunch of people that are kind of squished in the back. s48 00:03:51.120 --> 00:03:56.319 There's room here and here if you want to come in uh so we can fit everybody in. s49 00:03:56.319 --> 00:04:01.439 Um, before we start, here's a repo that we're going to be using. s50 00:04:01.439 --> 00:04:06.879 Um, so if you want to take a picture of the QR code, get clone that repo. s51 00:04:06.879 --> 00:04:14.400 Um, cd into the research desk directory and then run uh npm install or bonai. s52 00:04:14.400 --> 00:04:21.840 Um, I'll let you do that and then I'll walk you through through what we're going to be building today uh together. s53 00:04:21.840 --> 00:04:25.919 Um, still see some phones out. s54 00:04:25.919 --> 00:04:28.160 I know you're there's Wi-Fi issues. s55 00:04:28.160 --> 00:04:31.840 Hopefully, you can you can get uh you can get to the repo. s56 00:04:31.840 --> 00:04:33.120 Um Okay. s57 00:04:33.120 --> 00:04:33.520 Oh. s58 00:04:33.520 --> 00:04:34.320 Oh, I see you. s59 00:04:34.320 --> 00:04:36.320 Okay. s60 00:04:37.360 --> 00:04:39.360 Cool. s61 00:04:39.919 --> 00:04:41.280 All right. s62 00:04:41.280 --> 00:04:42.960 So, what are we what are we going to be building today? s63 00:04:42.960 --> 00:04:47.919 We're I'm going to show you how you can use cloud manage agents to build a variety of different applications. s64 00:04:47.919 --> 00:04:53.199 We're going to start with uh building a custom chat-like interface so that you can talk to an agent. s65 00:04:53.199 --> 00:04:54.479 We can say hello. s66 00:04:54.479 --> 00:05:02.880 Um, then we're going to augment that agent into a multi- aent uh ecosystem so that it can delegate tasks uh to sub agents. s67 00:05:02.880 --> 00:05:14.080 We're going to give it skills uh specifically a skill to be able to call um the SEC website and get SEC filings for individual tickers uh and analyze specific forms. s68 00:05:14.080 --> 00:05:24.160 Um then we're going to be able to use the these new enhanced agents to run more asynchronous um agentic tasks that can take a long time to run. s69 00:05:24.160 --> 00:05:33.600 Um and then we're going to bring it all together by enabling the sort of map reduce interface where we can talk to an agent and that agent can then delegate s70 00:05:33.600 --> 00:05:38.240 to many agents uh and do sort of like a map produce thing. s71 00:05:38.240 --> 00:05:49.360 um showing you how you can build a custom harness on top of CMA to orchestrate you know your own work with custom tools uh and manage multiseession um kind of interactions. s72 00:05:49.360 --> 00:05:57.759 Uh and then we'll finish up by I'll show you uh our deployments API that allows you to set up recurring work on a schedule so that you can run s73 00:05:57.759 --> 00:06:04.800 uh sessions uh on a on a given cadence to run kind of longer term analysis that keeps updating itself. s74 00:06:04.800 --> 00:06:07.039 Sound good? s75 00:06:07.199 --> 00:06:07.840 quiet. s76 00:06:07.840 --> 00:06:09.600 Okay, great. s77 00:06:09.600 --> 00:06:10.639 Um, all right. s78 00:06:10.639 --> 00:06:13.919 So, let's switch over to um a coding interface. s79 00:06:13.919 --> 00:06:16.160 I'm going to be using cloud code for this. s80 00:06:16.160 --> 00:06:19.280 You're free to use whatever coding agent you prefer. s81 00:06:19.280 --> 00:06:23.759 Uh, I don't think anybody's writing code by hand, so I'm not going to do that either. s82 00:06:23.759 --> 00:06:31.600 Um, there is a skill that we've loaded up in your repository that's going to walk you through the workshop step by step. s83 00:06:31.600 --> 00:06:41.199 Um, so if you want to rerun the workshop after I'm done, uh, later on when you have better Wi-Fi and better internet access, uh, you're able to do so, um, at home. s84 00:06:41.199 --> 00:06:46.560 Um, so the first thing you're going to want to do is I assume you've checked out the repository. s85 00:06:46.560 --> 00:06:49.199 You've cded into the research desk repo. s86 00:06:49.199 --> 00:06:50.479 You've run buni. s87 00:06:50.479 --> 00:06:54.479 We're going to spin up our uh, server with uh, bun rundev. s88 00:06:54.479 --> 00:06:55.440 I'm going to make this bigger. s89 00:06:55.440 --> 00:06:57.759 I'm assuming nobody can see. s90 00:06:57.759 --> 00:06:58.080 Okay. s91 00:06:58.080 --> 00:06:59.680 Is that better? s92 00:06:59.680 --> 00:06:59.919 Yeah. s93 00:06:59.919 --> 00:07:05.919 And then if you have questions while Gabe Gabe is presenting, feel free to raise your hand and I can run to you with the mic. s94 00:07:05.919 --> 00:07:06.960 We we have helpers. s95 00:07:06.960 --> 00:07:09.039 So if you run into issues, raise your hand. s96 00:07:09.039 --> 00:07:12.000 People will come um help you help you out. s97 00:07:12.000 --> 00:07:13.440 Um cool. s98 00:07:13.440 --> 00:07:18.080 So if you've gone this far, you'll basically be able to load this website. s99 00:07:18.080 --> 00:07:22.240 Uh and the first thing you're going to want to do is go to the setup page. s100 00:07:22.240 --> 00:07:25.759 Um everybody with me so far? s101 00:07:25.759 --> 00:07:26.160 Yeah. s102 00:07:26.160 --> 00:07:27.199 Okay. s103 00:07:27.199 --> 00:07:32.080 Um now we're gonna Yeah. s104 00:07:32.080 --> 00:07:33.199 Um, okay. s105 00:07:33.199 --> 00:07:39.039 Once uh once uh he takes a picture, we're we're going to start up uh our workshop skill. s106 00:07:39.039 --> 00:07:44.240 So in cloud code, you can type in uh workshop start. s107 00:07:44.240 --> 00:07:47.360 Uh and it's going to start running through the workshop for you. s108 00:07:47.360 --> 00:07:49.280 We're going to do it in six acts. s109 00:07:49.280 --> 00:07:52.080 Uh I apologize there's no intermission in between. s110 00:07:52.080 --> 00:07:54.240 Um I hope that's okay with y'all. s111 00:07:54.240 --> 00:07:56.000 Um but we're going to go through it. s112 00:07:56.000 --> 00:08:02.639 Uh Claude will uh sure we're going to go into auto mode. s113 00:08:02.639 --> 00:08:07.440 Um Claude Claude is going to keep track of each of the acts so that you can continue on. s114 00:08:07.440 --> 00:08:14.240 You can keep typing workshop next so that it you know it's going to keep uh progressing through the through the thing. s115 00:08:14.240 --> 00:08:20.560 Uh I'm going to ask Claude to do it for me but handhold handhold me and teach me what it's doing. s116 00:08:20.560 --> 00:08:25.199 Um and so the first thing it's going to basically make sure that our prerequisites are set. s117 00:08:25.199 --> 00:08:31.360 Um, it's going to copy the m.example file into amp.local. s118 00:08:31.919 --> 00:08:34.880 Um, oh no, stop. s119 00:08:34.880 --> 00:08:35.839 We're not there yet. s120 00:08:35.839 --> 00:08:37.360 Okay, it's a little eager. s121 00:08:37.360 --> 00:08:39.120 Okay, it's going to copy that file. s122 00:08:39.120 --> 00:08:41.919 That file um has a few values for you to fill in. s123 00:08:41.919 --> 00:08:44.240 You're going to want to paste in your API key. s124 00:08:44.240 --> 00:08:49.360 Um, you're going to want to paste in the SEC wants to know who you are when you're calling their API. s125 00:08:49.360 --> 00:08:51.600 So, you're going to have to put in your name and email. s126 00:08:51.600 --> 00:08:54.720 Um, this is just so that they know who's calling them. s127 00:08:54.720 --> 00:08:57.600 Um, and then that's basically it. s128 00:08:57.600 --> 00:09:05.600 Once, um, once you've done all that, um, it's going to, well, we're going to move on to the to the next thing. s129 00:09:05.600 --> 00:09:14.160 So, everybody with me so far, you've put in your API key into your.local, you put in your Edgar identifier. s130 00:09:14.160 --> 00:09:14.880 Show of hands, s131 00:09:14.880 --> 00:09:16.000 be a quick show of hands. s132 00:09:16.000 --> 00:09:20.080 Or how many of you are able to follow along who might have Wi-Fi or s133 00:09:20.080 --> 00:09:21.839 Okay, we got one one person at least. s134 00:09:21.839 --> 00:09:22.651 So, s135 00:09:22.651 --> 00:09:24.651 [laughter] s136 00:09:25.760 --> 00:09:28.560 MVP, pardon. s137 00:09:29.519 --> 00:09:30.640 Uh oh. s138 00:09:30.640 --> 00:09:32.240 Oh, for the repo. s139 00:09:32.240 --> 00:09:34.480 This one. s140 00:09:36.640 --> 00:09:39.040 Uh Rohan, you want to hand the mic? s141 00:09:39.040 --> 00:09:39.680 Yeah, thank you. s142 00:09:39.680 --> 00:09:41.200 Yeah, the slide with the credential. s143 00:09:41.200 --> 00:09:41.519 That's it. s144 00:09:41.519 --> 00:09:42.000 The setup. s145 00:09:42.000 --> 00:09:42.399 This one. s146 00:09:42.399 --> 00:09:42.959 Thank you. s147 00:09:42.959 --> 00:09:43.279 Okay. s148 00:09:43.279 --> 00:09:43.600 Sure. s149 00:09:43.600 --> 00:09:56.080 And then and then given given people don't have Wi-Fi plea please like stop us and ask questions so we can at least make this more interactive and help you understand concepts so that you can build with your own time after the workshop. s150 00:09:58.320 --> 00:09:59.120 Okay. s151 00:09:59.120 --> 00:10:03.600 Um I'm yeah I have a s152 00:10:05.519 --> 00:10:12.560 yeah this is the app spinning up uh when you clone and cd into research desk and run bun run rundev. s153 00:10:12.560 --> 00:10:15.519 This is a website you will you will get. s154 00:10:15.519 --> 00:10:16.959 Can I ask a question? s155 00:10:16.959 --> 00:10:18.399 Yeah, I don't see you but s156 00:10:18.399 --> 00:10:18.982 Oh, you're here. s157 00:10:18.982 --> 00:10:19.680 [laughter] s158 00:10:19.680 --> 00:10:26.480 Uh I'm wondering like you know so when you say cloud handle and manage the agents where is this running? s159 00:10:26.480 --> 00:10:27.839 Is this running on AWS? s160 00:10:27.839 --> 00:10:29.120 Yeah, that's a good question. s161 00:10:29.120 --> 00:10:32.800 Um so we we we're going to run it in our own infrastructure. s162 00:10:32.800 --> 00:10:44.000 Um today we we have like a onep offering uh that we run on but we also have cloud platform on AWS if you want to run on AWS using your AWS credits. s163 00:10:44.000 --> 00:10:48.640 What we also offer and maybe we can talk about this after this demo is self-hosted sandboxes. s164 00:10:48.640 --> 00:10:57.839 So if you want to bring your own sandbox through uh different providers like Burcell or Modal or Daytona or a bunch of others, you can do that. s165 00:10:57.839 --> 00:11:04.240 Uh even you know through Google cloud and and AWS if you want to run your sandbox in that uh environment, you can. s166 00:11:04.240 --> 00:11:07.440 Um okay, I'm going to continue with the demo. s167 00:11:07.440 --> 00:11:14.079 U the the first thing you the next thing you want to do is create push this button which is going to create um your agent. s168 00:11:14.079 --> 00:11:17.200 So it's going to create two things for us. s169 00:11:17.200 --> 00:11:22.560 um an environment which Priyanka talked about is a template of what the sandbox is going to look like. s170 00:11:22.560 --> 00:11:29.680 Um for this demo, we're going to give it access to a few websites that it can reach out to, mainly the SEC. s171 00:11:29.680 --> 00:11:38.399 We're going to add uh some Python packages so that it can reach out to to the SEC with you know some predefined uh Python packages. s172 00:11:38.399 --> 00:11:41.279 Um it's also going to create uh an agent. s173 00:11:41.279 --> 00:11:44.399 So it's going to create this very basic agent. s174 00:11:44.399 --> 00:11:46.079 Right now it has no tools. s175 00:11:46.079 --> 00:11:48.800 Um, it has uh just a prompt. s176 00:11:48.800 --> 00:11:51.040 Uh, well, it has the basic tools, but there's no skills. s177 00:11:51.040 --> 00:11:54.560 There's no sub agent coordination, nothing like that. s178 00:11:54.560 --> 00:12:05.760 Um, I'm going to tell Claude to um continue and implement the interface with which we can talk to this agent in our application. s179 00:12:05.760 --> 00:12:11.200 Uh, like Bianca mentioned, the main uh way you communicate with a session is through an event stream. s180 00:12:11.200 --> 00:12:19.519 So you listen onto that stream for all the events that the agent is pushing back down to you and you can use that uh an endpoint to push events back up. s181 00:12:19.519 --> 00:12:26.399 So really everything that happens in the communication for these more chat-like applications happens on that stream. s182 00:12:26.399 --> 00:12:40.000 So we can see here what Claude is doing is it's uh implementing the section so that when I press enter in my chat box, it's going to send a user message event with the content that I'm sending to that session. s183 00:12:40.000 --> 00:12:45.200 And then it's going to route the SSE stream back onto my browser from my back end. s184 00:12:45.200 --> 00:12:57.839 So it's going to listen onto um the events that are coming in the stream connection to our API and then reream them directly back onto the front end so that I can render them in a nice way. s185 00:12:57.839 --> 00:13:06.160 Um and once Cloud has done that, we should be able to go to this desk tab and just say hi to our agent. s186 00:13:06.160 --> 00:13:12.320 Um, and if everything worked hopefully great. s187 00:13:12.320 --> 00:13:15.040 Um, so we can see here now we're we're talking to our agent. s188 00:13:15.040 --> 00:13:19.279 If you wanted to build a chat-like application, this is where you would stop. s189 00:13:19.279 --> 00:13:22.160 Um, which is great, but we want to do more. s190 00:13:22.160 --> 00:13:26.880 The whole point of cloud managed agents is that you can do longunning asynchronous tasks. s191 00:13:26.880 --> 00:13:31.519 Um, not just for minutes, but hours, days, weeks, and even months. s192 00:13:31.519 --> 00:13:34.399 Um, so let's see what we we have to do next. s193 00:13:34.399 --> 00:13:35.360 Right. s194 00:13:35.360 --> 00:13:42.800 The next thing that we're gonna want to do is um Oh, Claude is very eager today. s195 00:13:42.800 --> 00:13:44.000 Um, okay. s196 00:13:44.000 --> 00:13:46.560 The next thing is we we want to augment our agent, right? s197 00:13:46.560 --> 00:13:49.839 We want to give our agent a skill so it can call the SEC. s198 00:13:49.839 --> 00:13:58.880 We want to use multi- aent coordination so that um our session can delegate work to sub agents that are more specialized and capable of doing some tasks. s199 00:13:58.880 --> 00:14:03.600 Um, and we want to basically move up the ladder of intelligence. s200 00:14:03.600 --> 00:14:08.639 Um so let's do that um by saying workshop next. s201 00:14:08.639 --> 00:14:19.440 Um now what claude is going to do is it's going to implement the provisioning scheme to upgrade our agent so that it uses uh more sub aents or any sub aents s202 00:14:19.440 --> 00:14:26.880 um is we're going to give it a skill a skill that lets it uh um talk to the SEC. s203 00:14:26.880 --> 00:14:29.120 We've described how to use the Python package. s204 00:14:29.120 --> 00:14:32.959 We've described that it has to use the Edgar identity to call. s205 00:14:32.959 --> 00:14:39.279 Um, we've we've described all of that and we're going to give it a custom tool uh for later in the workshop. s206 00:14:39.279 --> 00:14:49.920 Um, that custom tool is what we're going to be able to use to tell our head of research agent to spin off multiple sessions that we will coordinate uh for a sort of custom harness. s207 00:14:49.920 --> 00:14:51.920 So, this is what Claude is doing here. s208 00:14:51.920 --> 00:14:56.320 Um, it's updating its progress, creating this custom tool called dispatch analysis. s209 00:14:56.320 --> 00:14:58.720 We're going to come back to this in a second. s210 00:14:58.720 --> 00:15:00.480 um we give it a schema. s211 00:15:00.480 --> 00:15:08.720 Um it's going to update the the description of the main agent so that it creates a new version of that agent. s212 00:15:08.720 --> 00:15:13.279 Uh this new version will have um oh pager duty. s213 00:15:13.279 --> 00:15:22.160 Um uh this new version will have uh a few different sub aents uh that it can delegate work to as well as a reference to itself. s214 00:15:22.160 --> 00:15:28.480 So if the main agent ever wants to spin off a copy of itself with its same prompt, it can do that. s215 00:15:28.480 --> 00:15:41.600 Um this is important because uh sometimes you don't necessarily know what sort of tasks your session is going to want to do, but you have a general idea that you want to have sub agents that you know spin up and do work. s216 00:15:41.600 --> 00:15:48.000 Um so you can define a self reference to you know yourself that clock can then spin up a sub agent for. s217 00:15:48.000 --> 00:15:51.040 Um important this question has come up in the past. s218 00:15:51.040 --> 00:15:54.160 It doesn't copy the transcript of that session into the sub agent. s219 00:15:54.160 --> 00:16:02.560 It just copies over the tool description and the schemas uh and the prompt uh into into that sub agent. s220 00:16:02.560 --> 00:16:04.560 Um cool. s221 00:16:04.560 --> 00:16:11.759 So now if we go into here, we should see that there's a new agent. s222 00:16:11.759 --> 00:16:13.279 Oh, I think I need to provision them. s223 00:16:13.279 --> 00:16:17.519 So there should be a button in our setup uh that says staff the desk. s224 00:16:17.519 --> 00:16:21.279 When I do that, it's actually going to run Oh. s225 00:16:21.279 --> 00:16:22.880 Uh uh okay. s226 00:16:22.880 --> 00:16:24.000 I ran this before. s227 00:16:24.000 --> 00:16:27.519 Let me delete the skills so that I can recreate it for you. s228 00:16:27.519 --> 00:16:33.519 Um I was trying to do the magic uh cook show thing, but that's fine. s229 00:16:33.519 --> 00:16:35.199 Um okay. s230 00:16:35.199 --> 00:16:36.480 So we'll delete the skill. s231 00:16:36.480 --> 00:16:39.440 We'll ask cloud to reprovision it. s232 00:16:39.440 --> 00:16:47.519 Um it's going to create all those resources and then we'll be able to see here all the sub aents, the skill that we created, um a memory store. s233 00:16:47.519 --> 00:16:53.839 So for these sessions, we we we also support um longunning memory store that can be shared across sessions. s234 00:16:53.839 --> 00:17:01.040 Uh and we're going to use that so that different sessions can learn about different financial instruments and they can share that knowledge uh with each other. s235 00:17:01.040 --> 00:17:09.520 And at the end of this workshop, I'll show you how you can use that memory longunning memory to keep improving your agents long term over many many uh sessions, s236 00:17:09.520 --> 00:17:12.079 days, months, years. s237 00:17:12.079 --> 00:17:13.600 Um cool. s238 00:17:13.600 --> 00:17:23.360 So we should see here that the agent now in here um has the skill that we described. s239 00:17:23.360 --> 00:17:27.600 It has sub agents uh with their own prompts and tools and permissions. s240 00:17:27.600 --> 00:17:30.400 Uh it has a prompt and it has a model. s241 00:17:30.400 --> 00:17:39.840 Um and I think we're we're ready to use this new dedicated sub agent to do our first asynchronous longunning task. s242 00:17:39.840 --> 00:17:43.280 So let's uh tell Cloud to move on to the next thing. s243 00:17:43.280 --> 00:17:50.960 The next thing that we want to do is um we want to tell Claude not in a chat-like application but a more asynchronous thing. s244 00:17:50.960 --> 00:17:57.679 Hey, research this financial instrument and give me a report in a format that I define and I care about. s245 00:17:57.679 --> 00:17:59.360 And we're going to use something called outcomes. s246 00:17:59.360 --> 00:18:02.640 Outcomes is how we've created self-s supervision. s247 00:18:02.640 --> 00:18:06.080 So in an outcome, you tell Claude, "This is your task. s248 00:18:06.080 --> 00:18:08.640 This is your rubric to evaluate your task. s249 00:18:08.640 --> 00:18:18.559 It's going to go off and do that work and then when it thinks it's completed, it's going to ask a sub agent to look at the rubric and the work that Claude did uh and evaluate s250 00:18:18.559 --> 00:18:21.360 did it do it correctly and did it do it to completion. s251 00:18:21.360 --> 00:18:25.760 Uh if it does then great and if it doesn't it will try again. s252 00:18:25.760 --> 00:18:32.080 Um so here what Claude is implementing for us is basically this userdefined outcome event. s253 00:18:32.080 --> 00:18:44.080 So that anytime we start a session in our UI that I'll show you for a specific ticker, it's going to send this userdefined outcome with a specific rubric uh max iterations s254 00:18:44.080 --> 00:18:47.120 um and a description of the task at hand. s255 00:18:47.120 --> 00:18:50.160 And so then Cloud will go run that task for us. s256 00:18:50.160 --> 00:18:53.360 Um and so what does that look like in the UI? s257 00:18:53.360 --> 00:18:59.840 If you look to this scorecard, um I can tell Claude, hey, I I want to analyze Nvidia. s258 00:18:59.840 --> 00:19:03.760 Oh, please. s259 00:19:06.320 --> 00:19:06.880 Okay, great. s260 00:19:06.880 --> 00:19:10.400 So, it's going to kick off a session to do the analysis on Nvidia. s261 00:19:10.400 --> 00:19:11.919 Um, this is going to take a while. s262 00:19:11.919 --> 00:19:16.880 I'm going to do the cook show thing and show you a previous session um of of what happened. s263 00:19:16.880 --> 00:19:19.360 Uh, it's going to take about 10ish minutes. s264 00:19:19.360 --> 00:19:22.160 Um, but here's something I ran yesterday. s265 00:19:22.160 --> 00:19:25.840 Uh, the first thing we're telling Claude is here is an outcome. s266 00:19:25.840 --> 00:19:31.200 I want you to analyze Nvidia's most recent uh SEC quarterly filings. s267 00:19:31.200 --> 00:19:33.919 Um you're going to use Edgar identities. s268 00:19:33.919 --> 00:19:36.000 This is where your Edgar identities are. s269 00:19:36.000 --> 00:19:37.440 Um you have memory. s270 00:19:37.440 --> 00:19:45.919 So check if you've already done analysis previously and make sure that you use that instead of refiling and researching things that you already know. s271 00:19:45.919 --> 00:19:53.200 Um I want you to find the most uh recent form that I care about and run the analysis over that. s272 00:19:53.200 --> 00:19:55.120 Um and then I want two deliverables. s273 00:19:55.120 --> 00:20:08.960 I want you to write a note in in my memory um for that specific company and I want you to write a scorecard in a well- definfined structured output and I want you to save that in a specific file location. s274 00:20:08.960 --> 00:20:13.440 Um so then Claude went off to work started calling tools. s275 00:20:13.440 --> 00:20:16.559 It didn't find any notes in its memory store. s276 00:20:16.559 --> 00:20:22.400 Um it uh looked in a few different places and then it decided okay I know what to do. s277 00:20:22.400 --> 00:20:26.559 I'm going to leverage my sub agents to do the bulk of this work. s278 00:20:26.559 --> 00:20:35.360 Um I'm going to spin up a financial um expert analysis agent uh or extractor and then a risk evaluator analysis. s279 00:20:35.360 --> 00:20:39.840 And you can see all of that in the uh console here for full observability. s280 00:20:39.840 --> 00:20:45.679 Each of these lines here represents a thread and a thread represents the work done by a sub agent. s281 00:20:45.679 --> 00:20:54.400 And so each of the sub aents went off to work and then when they were done they let the main agent know hey this is what you asked of me uh and then the main agent woke up s282 00:20:54.400 --> 00:21:01.039 uh got that information from the sub agents and then uh filed everything it needed to do and created the deliverable. s283 00:21:01.039 --> 00:21:09.840 It created the JSON file that you know we specified uh and it made sure to update the memory and then the cell supervision started. s284 00:21:09.840 --> 00:21:15.520 So it kicked off a sub agent to verify that the outcome that we defined is being met. s285 00:21:15.520 --> 00:21:17.520 Um it looked at the rubric. s286 00:21:17.520 --> 00:21:23.679 It analyzed all the work that was done uh in the session and analyze the artifacts that it created. s287 00:21:23.679 --> 00:21:34.400 Uh and it decided in the end that this was satisfied that this you know scorecard was delivered uh that the numbers are well defined uh that the memory update you know was correct s288 00:21:34.400 --> 00:21:36.400 um and everything looked good. s289 00:21:36.400 --> 00:21:48.960 Um I don't I think this is still running but once that completes uh you would see a table here of each of those tickers uh because we can extract the report the scorecard from the session and download it. s290 00:21:48.960 --> 00:21:52.080 You put in a database for this demo we put it locally. s291 00:21:52.080 --> 00:21:55.120 Um but I think you get the idea. s292 00:21:55.120 --> 00:22:01.200 Um okay this is great but what if you want to do a hundred or or 10 tickers at a time? s293 00:22:01.200 --> 00:22:10.480 Wouldn't it be great if I could chat with an agent to build a custom harness that could then delegate work uh for each of the tickers into a different session s294 00:22:10.480 --> 00:22:15.039 and then when it's all done aggregate the results and give me uh a bunch of scores. s295 00:22:15.039 --> 00:22:18.080 Um so that's that's what we're going to do next. s296 00:22:18.080 --> 00:22:25.360 Um we're going to leverage that tool that custom tool that we created to build our custom orchestration harness in our server. s297 00:22:25.360 --> 00:22:34.400 um where when we talk to our main agent at some point that main agent is going to decide hey I want to spin up uh 10 different sessions to do work s298 00:22:34.400 --> 00:22:48.400 um we'll get that oh I skipped a step sorry um for we're going to update the skill uh first u so I I wanted to show you that these skills are are live artifacts that you can update s299 00:22:48.400 --> 00:22:57.360 um so you can improve the skill uh in this case uh we're adding the ability to look at a new form Uh and then when you do so any new sessions s300 00:22:57.360 --> 00:23:01.120 that the reference at skill will get that new information. s301 00:23:01.120 --> 00:23:03.679 Um so this is something that you can continuously update. s302 00:23:03.679 --> 00:23:06.320 You can look at previous sessions to see how they performed. s303 00:23:06.320 --> 00:23:12.799 We're going to see that at the end of this uh demo that that happens with with something we call dreaming. s304 00:23:12.799 --> 00:23:16.400 Um but yeah this this is what is happening. s305 00:23:16.400 --> 00:23:20.799 Um okay I'm going to move on to the orchestration bit. s306 00:23:20.799 --> 00:23:24.960 Um, so now that yeah, that basically had skills published. s307 00:23:24.960 --> 00:23:28.799 Um, I think if I go here, we should see s308 00:23:28.799 --> 00:23:30.080 any questions so far. s309 00:23:30.080 --> 00:23:31.840 Um, I know this is a lot to take in. s310 00:23:31.840 --> 00:23:35.679 Uh, Travis, there's a question over here. s311 00:23:42.799 --> 00:23:46.960 Um, and I I know someone came up to me asking about like how long it takes to get the credits. s312 00:23:46.960 --> 00:23:48.559 It should be only five minutes. s313 00:23:48.559 --> 00:23:53.600 Um, I know the form says one to two days, but you should see it within five minutes. s314 00:23:53.600 --> 00:24:00.320 Um, I was curious about the the data access uh from the SEC that the the research agent is grabbing. s315 00:24:00.320 --> 00:24:02.080 Does that require some sort of credentials? s316 00:24:02.080 --> 00:24:08.320 And are you going to walk through the part of how the how that credential injection or or brokering works? s317 00:24:08.320 --> 00:24:08.480 Yeah. s318 00:24:08.480 --> 00:24:12.320 So, for for the SEC, it just requires your email and your name. s319 00:24:12.320 --> 00:24:14.880 Um, for this demo, we just inject that in the prompt. s320 00:24:14.880 --> 00:24:18.159 But in a real production application, we would use something called vaults. s321 00:24:18.159 --> 00:24:24.960 Uh the way vaults work is you define a secret uh that you want to you inject in an environment variable. s322 00:24:24.960 --> 00:24:28.880 And instead of putting the real secret in, what we put is is a placeholder. s323 00:24:28.880 --> 00:24:37.360 Uh and then when Claude is doing work and it wants to use that environment variable, um it just injects the placeholder value. s324 00:24:37.360 --> 00:24:40.720 We at egress swap it out for the real value. s325 00:24:40.720 --> 00:24:45.520 only on the host domains that you specified are allowed for that secret. s326 00:24:45.520 --> 00:24:57.440 So if you say uh my super secret MVAR can only be used for the SEC and then cloud tries to use it for entropic.com we wouldn't inject we wouldn't do the swap but when it tries to call the sec.com s327 00:24:57.440 --> 00:24:59.919 it would do the swap for that s328 00:24:59.919 --> 00:25:10.960 is essentially like an API where you can use it on console where you just go create a vault give it a name you'll get an ID and then you essentially include that ID in your session as a way for us to authenticate with your MCPS s329 00:25:10.960 --> 00:25:22.960 your CLI you for example you could do like a Google Workspace CLI that you install on your um sandbox and then you can give us like your API key as your environment credential and again we'll we'll kind of follow the same method that s330 00:25:22.960 --> 00:25:23.919 uh Gabe talked about. s331 00:25:23.919 --> 00:25:25.919 So that that allows us to s332 00:25:25.919 --> 00:25:26.559 essentially s333 00:25:26.559 --> 00:25:30.159 bots also work with ooth credentials forps and they also work s334 00:25:30.159 --> 00:25:36.400 Gabe I don't know if you want to just show it on on console like the quickly like the vault creation. s335 00:25:36.400 --> 00:25:36.799 Yeah. s336 00:25:36.799 --> 00:25:41.760 Um so you you can create a vault here and then in a vault you would bundle credentials. s337 00:25:41.760 --> 00:25:57.360 Um so you can have MCP O um maybe you do notion you know we have a few predefined things for OOTH things um so whenever you reference a session you inject that vault ID and it will bundle all the credentials s338 00:25:57.360 --> 00:26:07.919 into your session and whenever it tries to use any of the domains that you specified in your credentials it will swap for the real um credential instead of the placeholder. s339 00:26:08.240 --> 00:26:08.559 Yeah. s340 00:26:08.559 --> 00:26:09.600 Um, quick question. s341 00:26:09.600 --> 00:26:16.880 What part of the capabilities in the cloud agents SDK and what's about it? s342 00:26:16.880 --> 00:26:18.640 Can you what what do you what do you mean? s343 00:26:18.640 --> 00:26:20.159 You have the agent SDK, right? s344 00:26:20.159 --> 00:26:22.880 I presumably this is built on top of the agent SDK. s345 00:26:22.880 --> 00:26:27.840 I want to know what part of the capabilities are in the agent SDK layer and what's outside of it. s346 00:26:27.840 --> 00:26:34.320 So the the agent SDK is packaged for uh like more more it's a headless cloud code version. s347 00:26:34.320 --> 00:26:37.279 So it's actually we've built a generic version of that. s348 00:26:37.279 --> 00:26:47.440 We've took taken a lot of inspiration about the benefits and lessons from the agent SDK, but we've rebuilt uh the harness to be a more generic not just coding specific thing. s349 00:26:47.440 --> 00:26:49.120 So it's sort of a fork. s350 00:26:49.120 --> 00:26:50.320 Uh yeah. s351 00:26:50.320 --> 00:26:50.720 Yeah. s352 00:26:50.720 --> 00:26:52.720 It's basically like its own harness. s353 00:26:52.720 --> 00:26:58.720 Agent SDK is basically like a cloud code in the form of a binary that you can take and run wherever you want. s354 00:26:58.720 --> 00:27:09.600 This is a fully hosted solution running in our cloud and we have our own harness that's heavily inspired by cloud codes harness and we frequently stay stay in touch for like any enhancements s355 00:27:09.600 --> 00:27:10.720 across the board. s356 00:27:10.720 --> 00:27:13.760 Uh I I saw another hand in the in the back over there. s357 00:27:13.760 --> 00:27:14.720 We have a question here. s358 00:27:14.720 --> 00:27:15.120 Yeah, s359 00:27:15.120 --> 00:27:15.600 couple more. s360 00:27:15.600 --> 00:27:16.159 Thank you. s361 00:27:16.159 --> 00:27:25.840 Uh I guess the question as you're orchestrating between multiple agents um is there some guard rails that those agents would not go into loops and like consume a lot of tokens s362 00:27:25.840 --> 00:27:28.559 um as they're generating or working together? s363 00:27:28.559 --> 00:27:33.200 Um some of that we leave up to you to orchestrate and figure out. s364 00:27:33.200 --> 00:27:43.360 I mean we're if you're building these sort of custom harnesses, there's nothing stopping you from using uh like other providers if you wanted to do a sort of multi- s365 00:27:43.360 --> 00:27:45.279 uh multi-provider system. s366 00:27:45.279 --> 00:27:56.240 Um, we are exploring the idea of what it means to do like budget tighter budget controls that you can say use no more than this amount of money per session, use no more than this amount of tokens per session. s367 00:27:56.240 --> 00:27:59.840 Um, but that's not something we offer today. s368 00:28:00.640 --> 00:28:01.840 Um, yeah, go ahead. s369 00:28:01.840 --> 00:28:03.279 Couple more hands. s370 00:28:03.279 --> 00:28:03.760 Question. s371 00:28:03.760 --> 00:28:12.960 Uh, earlier you were showing us in console kind of a transcript of the manager and then the sub aents and you could kind of go through and see what they were doing. s372 00:28:12.960 --> 00:28:19.760 if you were to come across maybe uh a mistake or like it was going down the wrong path or you wanted to change history. s373 00:28:19.760 --> 00:28:21.520 I mean obviously that's what's happened. s374 00:28:21.520 --> 00:28:28.159 Is there some way to correct it or to kind of go back to that point in time and kind of start over try it with some additional guidance? s375 00:28:28.159 --> 00:28:30.000 Is there any anything like that in here? s376 00:28:30.000 --> 00:28:30.799 You can steer it. s377 00:28:30.799 --> 00:28:39.760 So I could I could uh I think here if I say send event I could send a new user uh message event and steer the agent. s378 00:28:39.760 --> 00:28:40.799 Uh it'll queue it up. s379 00:28:40.799 --> 00:28:44.480 it's, you know, you can send that 10 messages at a time and it's fine. s380 00:28:44.480 --> 00:28:48.159 We don't currently have a way for like rewind and roll forward. s381 00:28:48.159 --> 00:28:52.960 Um, but that's definitely something that uh if if people are interested, we we would look into s382 00:28:52.960 --> 00:28:55.840 Yeah, session forking is essentially what you're asking for. s383 00:28:55.840 --> 00:28:58.640 We we've had a couple a few customers who have asked for it. s384 00:28:58.640 --> 00:29:00.240 That's it's it's on our road map. s385 00:29:00.240 --> 00:29:02.240 So hopefully we'll have it at some point. s386 00:29:02.240 --> 00:29:03.039 Thank you. s387 00:29:03.039 --> 00:29:07.440 Uh yeah, I see a couple hands here if you want to hand. s388 00:29:07.440 --> 00:29:08.559 Yeah, thank you Rohan. s389 00:29:08.559 --> 00:29:11.279 Uh so s390 00:29:11.279 --> 00:29:11.919 where are you? s391 00:29:11.919 --> 00:29:12.080 Okay. s392 00:29:12.080 --> 00:29:12.559 Okay. s393 00:29:12.559 --> 00:29:12.880 Yeah. s394 00:29:12.880 --> 00:29:13.455 Go ahead. s395 00:29:13.455 --> 00:29:14.799 [laughter] s396 00:29:14.799 --> 00:29:26.320 Assuming all of this is running in the cloud like like you know u you guys have any road map to get collective knowledge across these agents like let's say in this one particular application you have 10 users. s397 00:29:26.320 --> 00:29:27.440 Would would you mind speaking up? s398 00:29:27.440 --> 00:29:29.120 It's a little hard to hear. s399 00:29:29.120 --> 00:29:42.480 Oh, so so if you assuming all of this is running in the cloud and this application is being distributed across tens and hundreds of users and all of them are using it for the purposes of like you know doing this financial research. s400 00:29:42.480 --> 00:29:44.960 Do you have I know you guys have session logs. s401 00:29:44.960 --> 00:29:46.799 Is there a collective s402 00:29:46.799 --> 00:29:52.159 memory that you guys are looking into or even have a road map of? s403 00:29:52.159 --> 00:29:53.039 We have it today. s404 00:29:53.039 --> 00:29:54.480 We actually used it in this demo. s405 00:29:54.480 --> 00:30:00.399 So we use the a memory store um that allows you to share knowledge across sessions. s406 00:30:00.399 --> 00:30:09.760 Uh and you can see here that what's happening is um the agents are each populating individual information about what they've done so far. s407 00:30:09.760 --> 00:30:22.399 Um and then I'll show you at the end how you can consolidate that knowledge over time using dreaming so that you can continuously improve the memory stores as different agents run into different edge cases and collectively learn from each other. s408 00:30:22.399 --> 00:30:28.159 So just to follow up on this so you're storing this in anthropics infrastructure. s409 00:30:28.159 --> 00:30:37.279 Say if you want to delineate that and and have things running in AWS bedrock and I have my own like uh data clusters or something for sandboxing. s410 00:30:37.279 --> 00:30:39.120 Then you mentioned about self-hosting. s411 00:30:39.120 --> 00:30:44.799 How much of a lockin is this piece versus like how flexible is it? s412 00:30:44.799 --> 00:30:52.559 So all of this API based so you you can extract the memories and put them in different systems if you want to. s413 00:30:52.559 --> 00:31:00.640 We don't currently support out of the box other memory providers in the session but you would just build an MCP or a custom tool to do so. s414 00:31:00.640 --> 00:31:02.399 It's the way I would imagine doing it. s415 00:31:02.399 --> 00:31:02.559 And s416 00:31:02.559 --> 00:31:09.840 and also to add to the initial question about like having user level org level memory we do allow for attaching multiple memory stores. s417 00:31:09.840 --> 00:31:16.960 So you can imagine having a user level memory store and then having some sort of orwide memory store and then you bring both to each session. s418 00:31:16.960 --> 00:31:23.200 So you can start collecting user level as well as or level memories and then like Gabe said none of this is locked in. s419 00:31:23.200 --> 00:31:26.640 You can like take your data and go anywhere else if you wanted to. s420 00:31:26.640 --> 00:31:29.120 Everything is an API call. s421 00:31:29.120 --> 00:31:29.440 Yeah. s422 00:31:29.440 --> 00:31:33.200 Um I had a quick question as you walk through the agent workflow. s423 00:31:33.200 --> 00:31:37.360 I noticed that the grader is at the main agent level, right? s424 00:31:37.360 --> 00:31:44.640 like so how do you grade these sub aentic tasks that are happening and then how do you manage memory across those as well s425 00:31:44.640 --> 00:31:45.039 right s426 00:31:45.039 --> 00:31:59.279 that's a good question so uh there's two dimensions to this sub aents in the way that we've built them are meant to uh augment your context and leverage the capabilities of your of your session by collaborating between them s427 00:31:59.279 --> 00:32:08.399 uh the outcome definition happens at like an overall arching uh session level so you define the outcome and you say this is what I want you to accomplish. s428 00:32:08.399 --> 00:32:12.799 The main agent then uses you know uses sub agents to accomplish that work. s429 00:32:12.799 --> 00:32:16.720 It collaborates between them and it eventually says uh I'm done. s430 00:32:16.720 --> 00:32:22.640 Um there is no currently support for like a sub outcome definition. s431 00:32:22.640 --> 00:32:34.480 Quick followup like so as you have this dreaming feature right like so how do you learn like if the sub agents have some sort of you know uh you know offc course sort of actions then how do you sort of learn from that? s432 00:32:34.480 --> 00:32:38.320 Yeah, uh today the dreaming is constrained to uh memory stores. s433 00:32:38.320 --> 00:32:48.799 So I I'll show you that uh uh in a second, but basically the way dreaming works is it will analyze the transcripts and the memories created across multiple sessions s434 00:32:48.799 --> 00:32:52.960 uh so that it can update its memories so that future agents know what to pull. s435 00:32:52.960 --> 00:33:02.000 Um but you can imagine that we can continue to expand that for skills and for prompts and for agent definitions so that you can you have this continuous learning s436 00:33:02.000 --> 00:33:02.799 uh functionality. s437 00:33:02.799 --> 00:33:07.279 But right now we just have it for for memory stores. s438 00:33:07.760 --> 00:33:08.640 See a hand here. s439 00:33:08.640 --> 00:33:10.960 And then I'll I'll go back to s440 00:33:10.960 --> 00:33:11.600 Hello. s441 00:33:11.600 --> 00:33:12.000 Yeah. s442 00:33:12.000 --> 00:33:12.640 Here. s443 00:33:12.640 --> 00:33:14.080 Where where is here? s444 00:33:14.080 --> 00:33:15.279 Oh, hi. s445 00:33:15.279 --> 00:33:16.320 Thank you. s446 00:33:16.320 --> 00:33:29.279 Uh if you have your let's say company data in either I don't know traditional store or rag or whatever and you want to expose it to this type of agents through a custom CLI or MCP. s447 00:33:29.279 --> 00:33:32.720 what would be the safe path to put it in your environment? s448 00:33:32.720 --> 00:33:33.919 Yeah, that's a great question. s449 00:33:33.919 --> 00:33:35.440 There's two two ways to do this. s450 00:33:35.440 --> 00:33:47.519 If you are using if you can access that uh via CLI with an API or via uh an MCP, uh you would just create an MCP, give it uh give it the authentication s451 00:33:47.519 --> 00:33:51.039 vault uh and it would just be able to call your MCP directly. s452 00:33:51.039 --> 00:34:05.679 If you don't want to expose your MCP to the internet, we have a feature called MCP tunnels which lets you uh create a tunnel from your um network into ours so that you don't expose it to the internet and we would do it that way. s453 00:34:05.679 --> 00:34:17.200 If if that's uh too big a barrier to climb, we do support self-hosted sandboxes so that you can bring your own sandbox which has which is peered with your network and has access to your applications and services s454 00:34:17.200 --> 00:34:27.440 and the agent would just call the sandbox that you manage and host um and be be able to access your your resources that way. s455 00:34:27.440 --> 00:34:28.320 Yeah. s456 00:34:28.320 --> 00:34:31.839 Um I had a question on dependencies for the environments. s457 00:34:31.839 --> 00:34:35.359 So I saw something you could load in dependencies. s458 00:34:35.359 --> 00:34:40.159 Um is that at runtime or right before um it's created? s459 00:34:40.159 --> 00:34:40.399 It s460 00:34:40.399 --> 00:34:41.520 it's at creation time. s461 00:34:41.520 --> 00:34:49.599 Uh and then we'll cache it and reuse uh that environment across like re reinstantiations of the sandbox. s462 00:34:49.599 --> 00:34:54.879 So can you specify like internal dependencies or do you have to pick mpm package? s463 00:34:54.879 --> 00:34:55.200 Right. s464 00:34:55.200 --> 00:35:10.320 right now uh they are all they all have to be accessible via npm cargo uh pip whatever but you can upload files uh directly into um the session if or GitHub repos if you want to install them that way s465 00:35:10.320 --> 00:35:13.680 okay and one more thing uh deployments I see a tab for that s466 00:35:13.680 --> 00:35:20.720 yeah yeah I'm going to get to it I'm going to continue to the with the demo and then we'll we'll do more questions in a