Agent Speedrun — Elizabeth Fuentes Leone & Sandhya Subramani, AWS

AI Engineer · 59 min · 717 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:12Uh my name is Sandy and this is Elizabeth.
00:16Um and we have Mike and we have more people helping us today in case any of you have questions with um setting up your workshop environments and running.
00:27And everyone here is an expert.
00:28everyone helping us.
00:30We're very excited to get started today.
00:33We want to get all of you on board right from ideating an agent, building it, adding a lot of customization and then deploying it.
00:46So we want to start with the simplest form of an agent.
00:51Um, and we for this we're using Strand's agents, which is an open- source agentic harness.
00:58And this was actually built by AWS.
01:02Uh, but this is fully open source.
01:04And so let's walk quickly through it.
01:08I'm going to give you a quick introduction on what we're going to be doing and then we can dive straight into the code and all of you will have code that you can run, take home with you and you can run for two days after this, right?
01:19So let's get started.
01:20What is trans agents?
01:22Strands agents is an open-source agent harness which means you can you bring your own LLM.
01:31You can use any LLM provider model provider and you can build an agent harness around it and you can control it end to end.
01:39You can customize it.
01:41You can add your own tools to it.
01:43There's this is fully open source.
01:45So there's community supporting this on GitHub and AWS maintains this and it supports MCP protocol.
01:53It supports A2A.
01:54It's fully open and flexible and you can deploy it on any managed service.
02:00You can do it locally.
02:01You can host it anywhere.
02:03Um now like I said you can use it with any model.
02:07It works natively with uh in production with AWS but you can bring any model you want and you can run it.
02:14uh it's also got guard rails and observability built in.
02:18Uh now if when now when I say you can run it on uh a um Amazon bedrock you can with agent core you can also use any of the other services
02:29but you can also use a different provider you can also use your own homegrown uh hosted model if you want Ellie.
02:39Yes.
02:40Uh let's and because in this workshop you are going to put this agent that we are going to build in production.
02:47We are going to use agent core and agent core is a service.
02:51Amazon metro aent core is a service dedicated only to put agents in production.
02:59So inside aent core we have a core runtime.
03:03that is the runtime the container where you are going to put this agent and doesn't matter if it's trans the framework if the launch chain if the crew AI
03:14doesn't matter the framework that you're using you can put this aentic application inside the runtime the container and you are going to deploy this engine in one single life on the CLI
03:25and when you are using runtime this runtime have sessions and the session can run up to 15 minutes up to 8 hours.
03:36So you can have a session built in inside this container without any uh configuration because it's inside there and you have observability built into so you only have the code
03:51you uh install this code or put this code inside this container and you are going to have memory and observability and when I say observability is not observability about
04:03the RAM or what is the behavior for this container.
04:06container is about agentic observability.
04:08How how many tokens I spending in this application?
04:12What is what did what tool is using this agent?
04:16What happened inside my agent loop?
04:19You are going to able to see all of that inside the agent core observability inside cloudatch in AWS and we have a core gateway.
04:31So sometimes we are having a lot of tools a lot of MCP for our aentic application.
04:38So you can put all these tool of this MCP behind a aent core gateway and a core is going to be is going to deliver like a API or API is not the word like a endpoint
04:50that you can invoke inside your agent loop and this endpoint is going to be able to do like a semantic search for all your tools for all your MCP.
05:01So only one endpoint for all your NCP for all your tools and it have semantic search.
05:08So it help you to uh deliver the best tool for your agent loop and we have agent core memory because when we create a agent we have shortened memory.
05:22The short-term memory is built in the runtime.
05:25You already know that.
05:26But what happened when you want a a long-term memory, you know, a memory that can last more than eight hours.
05:34So you can create a a core memory.
05:37So when you uh create a core memory, you are going to have a semantic memory too.
05:43So agent core build for you in three different types of memory.
05:48So you are going to have like I don't remember the name of the memory but you have three different prompts capable to build the three different memories.
05:57So every time that you retrieve your agent loop you can retrieve this memory inside the agent called memory that is completely built in.
06:07You don't have to build the vector store the database.
06:11You don't have to build anything.
06:13You only have to say hey a core I want a core memory.
06:18So it's deploy for you the memory the three different types of memory custom tree memory and no building memory and you can create your own custom memory.
06:27You you can you put your custom prompt to build your own memory.
06:32So every time that you retrieve the information for this agent, you are going to have your custom memory and there is a core identity and a bunch of more agent core but we only have one hour and in this workshop
06:45you are going to deploy your agent in agent core runtime and I already say that and yeah so agent core runtime have uh give you all the that you can put your code and deploy to production.
06:59You have the enterprise grade security of course because it's running on AWS.
07:04So it's completely secure of course if you are doing a good deployment if you don't share your keys everything because it's inside AWS and we don't use your data to train models
07:17is in your account and it's only using there and scale automatically.
07:24So you have your container and you don't have to manage the scaling because runtime can scale automatically up if you have a lot of session if you have recession
07:35is scale down.
07:37Thank you.
07:38So today um this is what we're going to be doing.
07:41We're going to be starting with a basic agent.
07:43What is an agent loop?
07:45Um how can we build um an actual system with it?
07:49um and some ideas, deeper dive ideas about what actually makes a good agent, right?
07:55So, talk about hooks, talk about skills, steerings, and then Ellie's going to be talking about um session management, and she's going to be covering how to actually deploy to agent core.
08:06We also have evalentic frameworks as part of the workshop.
08:10If we have time, we will do it.
08:12But because we want to take it slow, we want to take it easy.
08:15We want to make sure everyone understands the basics before we get into the advanced concepts.
08:20But if we don't have time, you can still use this link.
08:22This is still valid for two more days.
08:24So you can still use it.
08:26Go back home, run it, complete all of the modules, and ask us ask any of us questions if you have any and we'll be there to support you.
08:33So uh first thing is uh go to this workshop link if uh we have so this is enabled for about 250 participants.
08:41If we have more and some of you run into problems we have a backup workshop link but for now uh log this website and it should take you to a link.
08:50Uh Ellie can also walk excuse me walk you through the steps if necessary.
08:55Um so this is going to take you to an account.
08:58Does uh everyone have the previous link?
09:00Do you want me to wait a couple of seconds?
09:03It's on there.
09:03It's on there.
09:04I think it's also there on the next few slides.
09:06I think uh so what what are we going to be doing today?
09:10We're going to go to this workshop studio account.
09:13Uh you don't have to so you don't have to create your own AWS account.
09:17Uh these accounts are already created for you and there is going to be like a Jupyter notebook format.
09:23Uh most of the code is already in there.
09:25We can customize it.
09:27You can run it.
09:28You can download it.
09:29You can run it in your own environments but for this uh session it's all there for you.
09:34So you can directly use and run it from there.
09:37Uh this is free of charge.
09:38You will not be using any of your accounts.
09:41It's not going to ask for any credit card.
09:43So just go walk with us and do it step by step.
09:46If you have any questions all of us are here to support you.
09:50Um so like I said first thing join this workshop link.
09:57So when you go to this link you are this link is going to uh give you a website.
10:05The first thing that you need to do is put your email email one time passport OTP.
10:11So then it's going to send you a email with uh I don't remember how many numbers is that yeah you need to enter your email and then you are going to receive the one time passport
10:25and you need to put that password inside the the window as similar to that
10:30and if any of you have questions please raise your hand and one of us will come find you we'll work with you
10:38and in this workshop when you start to either uh you are going to be able to use anthropic models through the Amazon bedroom API and I think I'm not sure I'm think that you are going to be able to use the new open AI models too and of score all the Amazon models that we have that you
10:59can invoke through Amazon API and uh you are be able to use a Amazon better agent core for free it's not so expensive okay I I know that and I know that people is having problems sometimes to use the new service because yeah it's normal to have I don't know
11:18uh it's money but if you try Amazon B aent cord in your account you are going to see that it's not so expensive and then you need to approve the next the next window is
11:33is everyone with us right now anyone struggling at any step so far you do you need help Sorry.
11:42Do you are you having trouble signing in?
11:45Um, so you're you've opened that workshop link and then did you type in your email address and your password?
11:57Okay, I think Mike can help.
12:01You are okay.
12:02I need to see
12:03everyone else.
12:03Okay.
12:04Any other
12:05problems?
12:05Is anyone struggling with anything?
12:08We might be having a little trouble with wifi.
12:11Who would have thought that a conference would have trouble with wifi?
12:14A lot of people are getting on.
12:16Are you going to be stepping through some of the things on stage there?
12:20Um what
12:21you going to are you going to run through the workshop stages on on stage?
12:25So if people can't do it, can they watch
12:28at the end of the session?
12:29Uh we are going to share with you all the slides.
12:32This is a Jupiter.
12:34This is a repository that you can clone too.
12:36you know this this uh this code everything that we are going to build here is not only available in the virtual studio.
12:43So because this virtual studio is going to be open for two days so probably it's better that we start to show what is inside and you can do this
12:53uh for your you know for your business
12:55and you can do it and you know I can I then when we finish the session I going to put this workshop for three days.
13:02So, so you can be you can use all these for free for 3 days.
13:08Yeah.
13:08And may the Wi-Fi gods be with us as well today while we run it through with you.
13:13Oh yeah.
13:14Um so if it's just network issues because of which some of you are not able to follow along, we will try to go step by step again hopefully with the Wi-Fi is working for us
13:26and um any other issues apart from network issues with signing up, signing in, anything at all.
13:31Yeah.
13:31Right.
13:32So before to start please take a picture to this.
13:35Yes.
13:35Because if you have problem with the first one you know it's not a problem of internet.
13:40If you are having problem hey there is not enough account is because the first one is running for account.
13:46So you can use the second one.
13:48So we have to uh separate environments deployment.
13:51Yeah.
13:51So let's get started.
14:06Okay.
14:06So, all of you should be seeing this right now, correct?
14:11No, that's not
14:12this, right?
14:13All of you should be seeing this.
14:15This is going to be on the left is going to be your stepbystep work um your work through on what what we're going to be building, how we're going to be building it.
14:26the setup each of the modules.
14:29So this is for you to follow along if you're ever stuck how to actually start running this.
14:34Right?
14:34So this gives you a full walkthrough.
14:37But if you want to get started with the code, click on this over here, the agent speedrun idea to code, right?
14:44Click on that.
14:45Scroll all the way down and you will see the code editor URL.
14:50Everyone with me?
14:51Click that URL.
14:54The people who is having internet.
14:56Yes, everyone who's got internet click like click on this button over here.
15:01Scroll all the way to the bottom to the code editor URL and then open that and this should pop up module one for you.
15:09So this is module one and you can close all of this.
15:14The first thing you will have to do is select your kernel, right?
15:18And which kernel do I select?
15:20go to the same workshop studio here under setup when you get started and you go to your next steps it's going to actually show you here right so select kernel python environments
15:33USR bin python 3.13 copy that and in your select kernel paste that over here and it should close it and pick it up for you okay so you You need to do this for every
15:52module that you're running.
15:54Now, what are we building today?
15:55Is everyone with me?
15:56Is anyone not with me?
15:59Oh, a bunch of you because of Wi-Fi.
16:01Is there any other
16:02just Wi-Fi?
16:04What do we do?
16:05Do we Okay, we're just
16:09Yes.
16:10Yeah.
16:10Thank you.
16:11So, if that's the case, then we're going to start walking through step by step.
16:15It's fairly simple and you can definitely go back and try it.
16:18Okay.
16:18So I'm just going to go with it and feel free to ask us any questions.
16:22So what are we building today?
16:23Today we're building a customer service agent and we're building that using Strand's agent which is an agent harness framework and how that works is it's basically using an agent with the loop.
16:38So the user sends in their input and their context and then you have an LLM which you can provide right because uh this is works natively with Amazon.
16:48We have a bedrock hosted model but you can have so if you go to strands agents over here this is the strand agents and you can and you look at model providers
16:58you can use any of your favorite model providers you can use your llama and if you want to host it locally and this shows you how to do it all you have to do is
17:07call the model here define the model and then under agent you just have to specify model and your model name you can do this with open AAI this supports all of your different LLMs your favorite LLM times use what you will
17:19um hugging face all of your favorite things right so use your favorite uh model bring your own uh API key and then connect it to the model and then we we are ready to get started
17:30so we are building um a simple customer service agent and what how what makes the agent powerful is the agentic reasoning loop which means when you give it when the user asks a question and says hey I have this trouble, I have this problem.
17:49The agent receives that input.
17:52The agent gets invoked.
17:54The agent calls the LLM and the LLM then reads the question and calls a tool.
18:01So we want to define a set of tools because tools are the way by which we give an agent agency to execute a function because LLMs are mostly just going to be predicting the next token.
18:15We don't want to really rely on the LLM's base knowledge of what the answer is.
18:20You want to ground it to a source of truth, to a source of information.
18:24And this information can be stored in your database.
18:28This can be stored as a file.
18:30This can be stored in any different format.
18:32And that can be written inside of a tool.
18:35And that tool can be given access.
18:38The agent can be given access to that tool.
18:41And the agent decides when to use that tool.
18:44And the agent can have access to a bunch of different tools.
18:47And the agent will figure out, it will rely on the LLM, the LLM's capability, the LLM's reasoning capability to figure out what to which tool to use and call the tool, run the tools response
19:00and potentially again, you know, the agent loop can get completed and then it would send a response.
19:05Now, another amazing thing about using an agent harness is because you can support a bunch of different models, you can swap out whichever model you want without really having to change the system prompt.
19:16You can if you want to, but you don't really have to change the system prompt or change the structure or the architecture of how everything works.
19:24This is it.
19:24And then you can try out with any model you want and then it still works.
19:28It's still good to go.
19:28So you can even compare your models different models performances against the same uh boilerplate code.
19:35So it's super powerful and then you get a response.
19:37Right.
19:37So how can we get started with this?
19:40Uh first thing is so uh in your module one we've got um we install the requirements.ext file.
19:49Then we define tools.
19:51Is this big enough for everyone to see?
19:53Do I need to zoom it in further?
19:55Okay, good.
19:55I'm seeing thumbs up.
19:57So what are we doing here?
19:58because we're doing customer service bots.
20:00First thing is we need to have data.
20:02So we've create mock data of two customers and their account orders.
20:07So they've ordered something off of a website and then they want questions.
20:12They want to get refunds on whatever they've ordered.
20:14So how are they going about this and how can we build an agentic system that can offer solutions to these customers?
20:22So they have these orders.
20:23Uh it's just like a dictionary.
20:24Okay, so these are the customers and these are the order statuses and we define tools.
20:29Tools can be written it can be uh it can be a functional code but it can also be uh like an NLP type of a tool that you can have generated.
20:40The way to define a tool is by using the at tool decorator function and you write a code.
20:46So here we've got a simple tool that's doing a lookup on customers.
20:50That's just doing a get customer ID.
20:52Then we've got another tool that's just doing get order history.
20:56The third tool we've got is process refund.
20:59Right?
20:59So in each of these cases, the tools can look very very different, but they're all the same format, all the same pattern.
21:07They all just have the add tool decorator.
21:09So let let's run this.
21:13And after that, what we do, and this is our initial code for creating tools.
21:19Now what we do to create an agent, we just need two basic things.
21:22One is your system prompt.
21:25The second is your LLM, your model.
21:29In this case, because we're using native uh bedrock models, hosted models, we don't really need to specify the model.
21:35We're just using the default model.
21:37But like I showed you in the on the strands website, you can establish whichever model you want and you can add your model here.
21:44The third thing you want to add apart from this is your tools.
21:49So you will call the agent method.
21:52Write the tool three tools that we've just defined and then create the system prompt which says you are a customer service agent.
22:00Um and these are your guidelines.
22:02Always verify customer using look up customer before taking action.
22:07Use tool data to answer questions.
22:09Don't ask the customer for info you already have.
22:11So in the system prompt you tell the agent like you tell the LLM how to use the tools that it's got access to and what to do with it.
22:21So that's how your LLM and your agent understands what needs to be done with it.
22:26So here now when we run this agent we just say hi I'm customer so and so can you check my recent orders and so now it's saying sure let me pull up
22:38um your account and the order history.
22:40So it's invoking tool one.
22:42It's doing the look up on customer.
22:44Then it's calling tool two which is get order history and it's saying ah okay this is a summary of your recent orders.
22:50And then it's saying hey is there anything else I can help you with?
22:53Now this is awesome but how can we figure out what's actually happening under the hood?
22:59Because a lot of times one of the problems we have is my agent is doing things and I have no insight into what it's doing.
23:06Where is my control?
23:07How do I get that?
23:08Right?
23:08So let's inspect that agent loop.
23:11So what we So this is how it works, right?
23:14And as we saw in the previous diagram, the user sends in a message, goes to the LLM, the LLM does the tool call, we get a tool result, and finally we get a response.
23:25Now we can look at the agent messages agent message.
23:29And so this is just code to see what the messages look like.
23:33And so this shows us the handoff.
23:35So this says the total number of messages for this is what the user is saying.
23:39This is what the assistant does.
23:41Then this is the tool call one tool call two.
23:44Then there's nothing else that the user has done.
23:46And so the assistant says okay here is the response.
23:49So this these are the metrics.
23:51These are number of tokens that it used.
23:52We can use more metrics.
23:54You can add more metrics.
23:55Do what you will.
23:56Additionally uh we just want to do something simple here, right?
24:00Because it's simple walk through.
24:01But what we can do is we can also have multi-turn conversations.
24:05This is just a single conversation.
24:07You can also run your multi-turn conversations where you have an extended chat with your agent.
24:12So for that you can also try these right.
24:14These are other samples that we've given.
24:16You're free to write your own if your internet is working.
24:19I hope it is.
24:20Is it?
24:21Um but also what you can do is we've created chat.py under the same module and we've got chat.py Pi file for every single one of these modules.
24:33So when I click chat.py, this is the exact same underneath underlying code except we've enabled it for multi-turn conversation.
24:42So you do the same thing enable um you just have to so when you go here what you can do is you can open your terminal and you just have to copy
24:56these two lines because we're already in this specific folder and then you pip install your requirements over here and then you open a thing and then you can chat with it directly if you want to.
25:07So then you can say, "Hey, blah, blah, blah."
25:10So you can just type, right?
25:12Um, and you can have multi-turn conversations with it.
25:14It's just going to reply and see there we go.
25:17So you can continue to have multi-turn conversations with it, the exact same co under underlying code, uh, exact same tools, and it's just able to talk to you.
25:2660 lines of code, less than 60 lines of code.
25:29Uh, so that's basic agent.
25:31Are we good?
25:35Okay, cool.
25:36Awesome.
25:37Second part, how do we make this agent better?
25:40What else can we do?
25:41This is this is basic stuff, right?
25:43Anyone's who who's been looking at blogs or just checking online would probably know how to do this.
25:49How can we make it better?
25:51Let me introduce you to what we call hooks.
25:54Now, hooks are amazing because let me first open that.
25:59Let me first open that code there.
26:01Okay, I'm going to close this.
26:03Now hooks are a way by which you can enforce certain programmatic uh commands and functions to take place.
26:16So before so what you can do is in this entire process where your from the time your agent is just getting invoked it's doing its own tool calls and it's getting responses and sending it out.
26:28what you we want to be able to control it, right?
26:30Right from the time that the agent is being invoked to the like before the tool is being called or maybe once the tool is being called and then it gets a response.
26:41We want to be able to have a lot of control over it.
26:43We want to potentially be able to validate who the who's using it like who which uh who are the customers that are probably talking to this or what type of questions are they answering?
26:54Is it even supposed to be be entertaining these types of questions?
26:57Do we want to have rate limits on the type of tools that are being called and the number of times these tools are being called?
27:03How can we do that?
27:04So when you have a hook, you can you can introduce your code at this junction where you can have an assert function or a different type of function that allows it to do to ensure that certain checks go through before it proceeds to your
27:21final output.
27:22So you can do it before invocation, you can do it after the LLM call, before the tool call and then you can decide whether you want to approve of it or cancel it.
27:31And then you can even do it after tool execution, before it sends the response.
27:36So you can do it at all of these spaces, but you can also do it at more locations.
27:40And um this is just an example.
27:42If we go to strands agents, this will show you all the different ways you can enforce hooks.
27:47Um so this is this where I am at?
27:50No, I keep I'm going to close all of these.
27:52Um, okay.
27:54So, uh, in hooks, what are we doing here?
27:57We've got the same data that we're calling.
27:59Let me run this first.
28:03Okay.
28:04Go to the notebook.
28:07This is not the notebook.
28:08No.
28:08Oh, this is a customer service tools.
28:10Okay.
28:11Um, yeah.
28:12Yeah, I'm on the wrong one.
28:17Okay.
28:18I'm on the notebook again.
28:21We have to install requirements and choose our environment.
28:26So how does this work?
28:28Now we want to import strands.hooks and we want to import hook provider hook registry and where we want to call these hooks.
28:38So in our case we want to do it before invocation and also before tool call.
28:43And I'll tell you what we're doing.
28:44We're trying to create a rate limiter hook which ensures that for one user request the number of tools that are being invoked is not more than three in case you have
28:57um an LLM that's just going crazy and that's continuously invoking your tools that's going to cost you a lot of tokens and you know it's not working right so you want to create a hard stop rate limiter and say hey even if it's not working it's fine stop it at three right so now that's what we're doing
29:13how can we create that rate limiter over here.
29:16So first what we do is we define the rate limiter hook and we want to say okay these are the maximum number of calls we're creating a default number
29:25and then we have to add it to the registry.
29:27So over here we add it and say okay be before invocation event and then we want another call back which is before tool call event and a call back means that's when it would get called
29:39that's because you're defining it over here.
29:42So then we have this counter which that which at before every response like every new time the user is asking sending in a request this gets reset to zero because you don't want it to continue running that
29:56u rate limiter and we define a second one called check second function called check that checks the number that's basically like a counter it counts the number of times your tools are being invoked
30:08so that's what this does and then This looks at the max calls and then it just calculates that.
30:15So we define our rate limiter over here and this is the main function that we've been looking at.
30:22One extra line that we added is the hooks and we say rate limiter hook max calls equals three.
30:29And if we run it, it's calling these two and it's saying okay tool call one tool call two.
30:36Here's a summary of your events.
30:38This is what's happened.
30:39But let's actually test it.
30:41Let's try and break it and see if the rate limit is actually working.
30:44Right?
30:45So here I'm doing max calls equals true.
30:47You want to lower the limit to the trigger.
30:50So over here it's calling it so many times, right?
30:55And so it's actually blocked it.
30:57So here it's saying block lookup because it's exceeded the number of times.
31:01So when your agent starts to go haywire and we all worry about oh my god is my agent going to take over everything, right?
31:08It won't because we have something called hooks.
31:12Is everyone with me?
31:13Are we good?
31:14Is anyone able to run it with me?
31:16Am I going too fast or are we just watching us?
31:19That works.
31:20Oh, okay.
31:20Good.
31:20We have at least a few people.
31:22I I would like to add something to the hooks.
31:24Sometimes, you know, hook is better because when you're using a LLM with some tools, the LLM are probabilistic.
31:32Yeah.
31:32You know they when you put a prom inside your agentic application the prom is like a suggestion you know it's only about the probability to give you an answer.
31:43So when you use a hook you can be more deterministic.
31:47So you can put code inside your aent loop so you can put code a python code before or after the invocation.
31:57So you can stop uh I don't know in this example you stop a a lot of more than three tools invocation but for example if you want to restrict
32:07a I don't know some credit cards number something like that you can put a hook a code capable to understand some things so yeah hey you can receive this information so I going to block this with this deterministic
32:22uh function so you don't receive that so the model the model never is going to receive that information.
32:29So you block the call so you don't waste token.
32:32So you don't have to invoke the model again.
32:34You don't you don't waste tokens because you are stopping that.
32:38You send a hook.
32:39And if you don't want to send some type of answer to your customer service, you can stop that answer with after call hook as well.
32:50Exactly.
32:50Thank you.
32:54Okay.
32:54So you can try it out yourself.
32:56And again, if you want to do multi-turn conversations, go to chat.py in the second module with hooks enabled.
33:02You can still continue to chat with it.
33:04And you can play around, add different hooks at different places, see what works.
33:08You can try and block certain customers or block certain types of questions.
33:12Thing guard rails, right?
33:15So hooks are amazing and we want to have hooks as we're writing good building good agentic systems.
33:22Now, let's add one more layer of coolness to this project, right?
33:27Let's add skills.
33:28Let's add skills and let's find a way to steer your agent and your LLM and get them to respond in a fashion that we want it to respond in.
33:39And so, steering is I love steering.
33:41Steering is awesome.
33:42Um, so let's get to steering.
33:46Do I have any questions?
33:47Everyone good with the idea of hooks?
33:50Yes.
34:07So the so if we sorry
34:12oh okay okay okay um so the question that was Oh you have a mic now do you want to
34:17yes
34:18okay ask oh this is wrong uh I saw that in the example with the hook, it kept running the agent, but it just blocked the tool call.
34:26How do we stop the agent so that it it doesn't go into infinite loop and keep looping?
34:30And
34:31so, as we're defining our hooks, we can define the hook in such a way to even end the agent's turn if we wanted to.
34:38For demo purposes, we wanted to show you how it's working so that we can see it doing the rate limiting and we can see that it's not running multiple times, right, the calls.
34:48So what it's doing is it's trying to invoke the tool in this case because we said before tool invocation it just kept attempting to do it and kept stopping and but the LLM still continued to run the loop until the end.
35:00But if we want we can create it in such a way that we say okay if I see this just stop the turn and end it at that point.
35:07So the we can definitely customize just because we wanted to show you the end to end working.
35:13We made it this way.
35:14Um thank you for that question.
35:17Um, okay.
35:18So, skills and steering.
35:20Now, now this is a chatbot that's supposed to be answering customer service questions, but they could be asking a bunch of different questions that the agent may just not have access or the knowledge to answer.
35:33So, how do we give it that knowledge?
35:35How do we tell it how to answer it?
35:37We create what's called a skills folder, a skills file which contains potentially all of your information on your your your ground truth, right?
35:48Your uh single source of truth maybe on what it what actually what steps it needs to take.
35:54And we can add this as a markdown file.
35:57And this is a way by which so here what it do what it would do is when we add agent skills it would first go look up the skills folder
36:07and read all of the skills see which one actually makes sense and then it would use those skills.
36:15Um let me get to steering in just a bit but skills is a way by which we can give our let me go to the third one third module.
36:26So um I'm I'm just gonna run this first.
36:35Perfect.
36:36So first thing is when we create skills now we've got three types of skills that we've created.
36:41One is how do you process refund?
36:44How do you track orders?
36:45And what if people ask some types of account related questions?
36:49My password is not working or something a bunch of frequently asked questions.
36:53How can we get it to work?
36:55Right?
36:55So this one is on how to handle account issues.
36:58So here we are creating these different skills plugins and then we can directly call it.
37:05Um so when I open the skills folder here we can look at sample skills.
37:11So this is just a markdown file on skills for account troubleshooting which is giving it context additional context because in your system prompt you don't want to overflow it with all of the amount of information you have because those are also considered input tokens and those can get wasted
37:29unnecessarily needlessly.
37:31So when a certain type of question is being asked, we can give it context that is relevant to that specific type of question then it has access to all the information that it needs so that it can do justice and answer the question correctly.
37:45So in this case this specific skill talks about how can we do account troubleshooting?
37:51What does it require to reset a password?
37:53What do you do when the account is locked?
37:55And the same thing for like order tracking, right?
37:57What do we do?
37:58What are the steps that we need to follow?
38:00So these are instructions that we give to the agent beforehand and we write this based on what we want the agent to do.
38:08And so we create these different skills um files markdown files and when we run it apart from the system prompt what we give it access to is the plug-in.
38:22So we've now we're now adding yet another uh parameter here which is called plug-in and this has access to all of the skills plugins and the skills plugin points to
38:34the skills folder here.
38:36So now it's going to invoke whichever skills it needs for that specific type of question that's being asked.
38:43And when this runs and we someone says, "I'd like a refund for my wireless headphones," it's actually calling the skills as a tool.
38:53And then it's saying, "Okay, I've got your account pulled up.
38:55Here's what I found and here's how it's doing it."
38:58So, it's becoming more and more deterministic.
39:01It's got a natural order that it flows and you can invoke the context only at the time that it's required.
39:07And it's not you're not lo overflowing your system prompt with a lot of tokens every single time.
39:14Now this is skills but how many this is something that I've generally faced right with steering.
39:20How many times have we struggled as we're talking to our agents to give us an answer to a question that we ask in a specific format and it always messes up.
39:31It seems to get all of the difficult questions right.
39:34If I tell it this is so if I ask it okay what is my what is the refund I'm supposed to get it would give us a refund and say okay this is what it is but they also generally tend to talk too much and generate a lot of tokens and they you know just free flowing.
39:50So how do we give it how do we get it to respond in the format that we want?
39:54How do how can we ensure that it's giving us like a structured output if that's what we need or like hey stop using m dashes every single time do we want to keep telling it we want to be able to catch that
40:06deterministic the deterministically and ensure that it's doing it in a deterministic way.
40:11So for that we do something called steering which helps the agent which helps the agent steer itself in the direction that we want it to by giving it a buddy agent.
40:25So it works, think of it as like pair partnering, like pair programming or pair partnering where as it's generating an output, the buddy agent looks at what's being generated
40:36and then that buddy agent can then tell it, hey, no, this is not what you're supposed to do.
40:41This is the format that you wanted to do in.
40:43So your buddy agent is sort of checking and correcting and course correcting and then the response you want is what is the response that you're actually happy with.
40:52When you're deploying your code into production there is no room for egger in like format structure.
40:59You want it a certain way and steering can help you do that.
41:03Now I'm going to do I'm going to do this quickly because I want Ellie to talk about deployment which is how you can actually make sure your agent is helpful and useful.
41:12So um quick um quick idea of steering handlers.
41:18Now stirring handlers works the same way as hooks.
41:22You can um call it you can invoke it at different points.
41:25You can invoke it uh before LLM while LM is being invocated before tool call after tool call.
41:32very similar to hooks except at that hook point now you're also adding another agent over there that can do a quick check and that can buddy like sort of buddy work with this and then give it the output once so think of it as another guard rail if you will right so then you can do a tool
41:50call and then you can do this and this is a guide that can say oh you know what first look up this before you give me a refund or then you can say revise the tone of this message if it's sending out a message and then after that it gets sent.
42:03So now you have more control over what your agent is generating.
42:08And if we go back to steering.
42:13Okay.
42:14So again what do we want?
42:15We want to uh import the steerings uh uh we want to import steerings over here.
42:21And from here again the steering handler.
42:23And these are all the different places that we're doing it at.
42:26And you can do it in different ways.
42:28uh check the the strands agent documentation.
42:32It's the details are going to be in there but also in the workshop mentioned over here you will see all of the details.
42:39Now we create a deterministic handler.
42:42We tell it we give it rules and we tell it what it's supposed to do.
42:46And this can also be a hardcoded function.
42:50This can also be another agent that has an hardcoded function in it that works.
42:55We can get it to do you can do like um type enforcement if you want the output to be only of a specific type.
43:02You can do different types of enforcements over here.
43:05Right?
43:05So then you create a a steering agent and then this is all for steering.
43:11Right?
43:11So now you create a steering agent and you create the guard rail.
43:16So here the refund workflow handler has been defined and finally you create the guardrail.
43:23So here you're saying okay don't overpromise guarantee specific timelines don't blame the customer acknowledge the customer's frustration and so this is the tone guardrail handler that we're creating.
43:34So this is the second type of steering agent that we've created.
43:38Now what we do is in our final code apart from agent skills we just call these two different handlers that we've established and these handlers can be established at different points.
43:51So we can say steer after model if we wanted to do and we can tell it what to do in this case.
43:57But over here we can also tell it we can also do steering data.get we can create a ledger for it and we can choose to enable them at whichever part of the flow
44:09the agentic flow we wanted to do.
44:12And so at the end all we need to do is give it access to these two sterving handlers.
44:18it's going to be enabled and then when you run it, it's not just calling the lookup customer and get order history, but it's also we see that it's evaluating
44:34the agent response.
44:36So before you send out the final response to your end user, you can ensure and you can be you can do it with full confidence.
44:45you can say with full confidence that your model that your agent is most likely going to be responding in the way that you want to because we're establishing all of those control features.
44:57Um you can try it yourself and then I'm going to quickly give it over to Ellie because she's going to be talking
45:01because only have 12 minutes to put this agent in production but I want to I would like to add something to the steering part.
45:09you know uh there is a a open source library that is calling agent control that you can add to a stringent is there in the documentation with agent control
45:21you can create rules to the steering for example if I have a demo where I this is a travel engine so you can use hooks to block numbers for example it's one one to booking a room for more than five people
45:36you can use a hook a hook to block the room for more than five people.
45:41But if you want to steer the agent to block for more for five people, you can create rules.
45:48So every time that the agent is going is receiving more than five bookings, he's going to say if I receive six, I can steer the booking for two rooms.
45:59So you can create rules for that using the agent control library.
46:04And the rules you can host this rule in a database.
46:08So you can create a database for all your rules for steering the agents.
46:13So every time that you invoke this agentic application, you can go to the database and see the rules.
46:20So what is the good thing for this?
46:22So you only have to change the rules in the database.
46:25You don't have to change the code for the agent.
46:28So the agent only have to knows they have to know the endpoint for your database.
46:33So every time that you have a new rule, you can edit the database and that's it.
46:39And let's put this agent in production.
46:43So here we have the session manager, but I don't going to show you that session manager is when you want to save the memory using a framework.
46:53So but you can do that using n get called memory without the framework and everything is in the in the code and here in the five for deployment we are not going to use a jupy notebook
47:07because you can do that for jupy notebook but we are going to use the CLI the terminal.
47:12So here in this folder uh lose the folder we are having the same engine.
47:17So we have the customer service tools and we having the uh the skills we have everything here.
47:25So to put this engine in production you have to go to the workship studio here.
47:30This is the thing what we are going to do.
47:33We are going to use the AWS CLI through the terminal.
47:36We are going to use some one command or two commands in the CLI.
47:41We are going to take this agent then we are going to send this agent this agent code to a runtime on AWS and agent core is going to deploy this code inside a container
47:53but we are not going to handle that because agent core is going to do that for us and and we are going to have memory too without doing anything.
48:02So let's see let's follow the steps.
48:05So this is the ah something important when you build a gentic application and you want to put that code inside a core runtime the only different that you have to do for your engine is put this
48:19deconator the a entry point.
48:21So the MPP entry point is is the only different between your engine running in your notebook or your computer uh between production because the MTP entry point is the thing that the runtime when runtime see can see the MTP entry point ah
48:39everything behind the MTP entry point is the agent.
48:43So every time that you invoke this runtime, this container is going to run that code behind the empty entry point.
48:53Okay.
48:54So let's style this.
48:58So I go to the here.
49:01Sorry, let me open this.
49:11This is the library that I need to execute all my CLI codes and mean I hope I have good internet is running.
49:20Yeah.
49:20So in this environment you are going to able to deploy the agent and everything.
49:27But if you are running this locally you are going to be able to test the agent before to putting it in in AWS.
49:35And something that you have to do to have the the built-in observability, you need to do something in the console.
49:41So let let me show you this.
49:43So to go to the console, you open this part AWS console.
49:49So it's going to open the console that is running in this session in this uh in this uh account that you have.
49:57So you need to go to a core run not here aen core better aen core so you can see all the service that a core have harness because you can build a hardness with a few lines of code too runtime gateway memory
50:15and observability is just right here but you need to select something I don't remember what is the name but I'm going to show you let me see here here enable
50:28no that's not here configure so you need to go to configure and here you need to enable transaction search so this thing allows you to have the built-in observability
50:42inside the runtime if you don't have that you are not going to see the agentic observability inside your agent because cloudatch is not going to receive the information and another thing that you have to do let Save this
50:57is install put inside the requirement the observability library that I have the observability requirements.
51:07So for that sorry oh it's done.
51:11So we have this transm requirements because I'm going to use AM core and you need this one the AWS oper telemetry district.
51:20So if you are using another framework for example line chain or ci there is a unique library for that uh for that framework but you need to do some modification
51:32inside your agentic application for a strand you don't need to do anything because a strand is built in for AWS so everything is built but the observability works for other frameworks but you need to do some modification
51:44everything is in documentation so you can see it so we have the library let's go to So here I'm already there.
51:53So A and C. Great.
52:01Oh, so I going to create.
52:05No, it's okay.
52:06I I didn't save it.
52:08So my name I going to put Ellie because I am Ellie.
52:14And for here I don't going to create a harness.
52:16I going to create a agent.
52:18But I have my code.
52:20You know, if you if you choose agent, it's going to be a simple agent for you.
52:25But I have the code, so I can skip it.
52:27And it's going to create environment everything that I need to put this aent that I already have inside this container.
52:35So it's going to create a folder for me.
52:37Here is my folder, the aen core, everything.
52:40So you all these steps are here in the workshop.
52:44Oh my god.
52:44Five minutes.
52:45But I going to do it in five minutes.
52:47I'm going to put this aent in production.
52:49That's all right.
52:51Yeah.
52:52So, a core add to add my aent and what is here.
52:59So, well, it's taking a moment.
53:02When I put a is going to add create the the container that I need, but I already have this agent here in my computer because I wanted to show you something.
53:13the same engine that Sandy just showed you.
53:16This customer engine I have I have the repo here in my computer.
53:20So, let me put this bigger.
53:22This is the same again the in the five.
53:25Yeah, you can use a core dev agent core dev to test the a locally.
53:36So you can do this you can't do this in the virtual studio because virtual studio is a virtual machine that only allows you to run code in this VS code
53:47but if you are running this locally in your computer you can have this environment to test the engine.
53:52So this is the same agent and you know I have some sessions here too.
53:57You can see the amounts of tokens.
53:59If you want to explore what's happening inside your engine and you can do that locally but because I'm using Amazon Metro as a model provider I'm using my AWS
54:12credential and to use Amazon a core you need AWS credential if you are running this locally but in the Amazon brow and workshop you don't need that.
54:23So yeah, it's working.
54:25Yeah, it's the same exactly the same area and I didn't do anything.
54:28So let's come back to the
54:31Yeah, I have my everything.
54:34Yeah.
54:34So I I I did this.
54:36I don't remember if I did this.
54:40Nope.
54:40So CD
54:42early.
54:44So I going to add my uh code inside this aented application.
54:50Let me see inside this container.
54:53So agent core what I want I want a agent I going to uh click bring my own code that's it that's it and I you can use direct code deploy or you can use container
55:09I going to use that and you can bring your own API key to deploy this a using entropy open IO whatever you want but I'm going to use Amazon better I don't going to do anything here
55:21Enter.
55:22Enter.
55:23That's it.
55:23Now I going to move my code.
55:27And to move my code, I have here copy and paste.
55:33And okay, now what else?
55:37Ah, I need to create this.
55:40So let's see.
55:41Copy and paste.
55:42One minute.
55:45Wait.
55:46And I don't going to do that.
55:48So core deploy and one minute to deploy one minute to put this agency in production inside AWS.
55:58So it's running is going to validate it's going to check my credentials.
56:03So if you are familiar with AWS when you build everything inside AWS you need to create the am you need to create policy permissions you need to allows the service to connect with other service you need to do a lot of thing inside
56:17core is handle everything for me so you I don't need to create the policy I don't need to create permissions I don't need to create every anything so it's doing everything for me and you can you can see that ah and because I created that locally
56:31it's creating a cloud formation is a infrastructure as a code using CDK.
56:37So you can explore the CDK code.
56:39If you are not familiar with CDK, CDK is cloud developer kit.
56:43So you can create infrastructure as a code with a l a program lens one that you know for example Python, TypeScript, whatever.
56:51So it's doing that for me.
56:52It's creating it's creating everything.
56:54It's creating the deployment and everything.
56:57And you can do that you can see that in the console.
57:00So let's go to Amazon bedro runtime.
57:05Yeah.
57:05What?
57:05I can't see anything up.
57:07So it's it's creating the let me show the cloud formation.
57:17Here I going to see my stack.
57:18Ah no because it's a virtual studio account.
57:20I'm sorry.
57:20You can see that here.
57:22But when it start to put this inside the runtime I will see that here.
57:28There we go.
57:29So it's here and you can test the is is ready.
57:33Okay, I can test the end right away.
57:35So I can invote this engine here and you know this is super uh terminal everything inside the repo.
57:46You can find how to invoke this a using bo tree.
57:50So you the only thing that you need is the air and for this agent is working.
57:57Okay, I think it's working.
57:58I hope so.
57:59It is.
58:00It's invoking the agent.
58:01Yeah.
58:01So here you can find how to invoke this agent using the uh SDK.
58:07Is there is give me the answer.
58:09You can
58:10it it actually worked.
58:11I think is there anything else I can?
58:13Yeah.
58:13So you can invoke this agent using the SD explore.
58:16how to involve this engine using the SDK and to see the observability.
58:23You can go to the runtime.
58:28In here you have the observability dashboard and you can see the session the trace how many runtime that you running how many session do you have each session is a runtime a different container
58:43it take a moment you take about one or two minutes to see everything there I'm okay what yeah yeah I'm done that's it please take a picture for this
58:56and share it for the other Friends, here you can find all the information.
59:02Everything is here.
59:04The the deck, the work studio, some credits if you are lucky because our few about 20 credits for Eds
59:14and the Feda please.
59:16Firestar, we are amazing the internet.
59:18Nobody's not at all
59:20[laughter]
59:21and all social media and if you have an answer we are on linking, Twitter, Instagram.
59:31Please in the QR code is everything that you need to learn more about strands and a core.
59:37Thank you.
59:52[music]