WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=vJukHCIv7Ck NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/vJukHCIv7Ck.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:00.000 --> 00:00:11.360 Hi, today we are going to talk about how to stop AI agent hallucinations with five techniques beyond the prompt. s2 00:00:11.360 --> 00:00:16.560 Each one is a code change, not a prompt change. s3 00:00:16.560 --> 00:00:18.320 Let's see. s4 00:00:18.320 --> 00:00:26.200 Every time your AI agent responds, you are paying for the words going in and the words coming out. s5 00:00:26.200 --> 00:00:30.280 And your bill, you will see those calling tokens. s6 00:00:30.280 --> 00:00:31.320 Yeah. s7 00:00:31.320 --> 00:00:36.320 And the more tokens you are sending, the more you pay. s8 00:00:36.320 --> 00:00:46.920 And if what you send is not quite right, too much, or missing something important, your AI agent start to hallucinate. s9 00:00:46.920 --> 00:00:59.160 There are five techniques to help reduce tokens waste, improve accuracy, and catch failure before you submit them. s10 00:00:59.160 --> 00:01:05.199 And each one is a code change, not a prompt change at all. s11 00:01:05.199 --> 00:01:07.760 So, let's see it. s12 00:01:08.280 --> 00:01:12.520 First, we have semantic tool selection. s13 00:01:12.520 --> 00:01:18.400 You filter which tool go into context on every call. s14 00:01:18.400 --> 00:01:23.240 The model only see what it needs for that specific query. s15 00:01:23.240 --> 00:01:26.400 Second, we have graph rack. s16 00:01:26.400 --> 00:01:38.360 For precise queries like aggregation, counts, multi-hop reasoning, and you replace the text retrieval with a structured graph query. s17 00:01:38.360 --> 00:01:47.760 The model gets a compute a verifiable answer, not a sample as rack do. s18 00:01:47.760 --> 00:01:50.640 Three, multi-agent validation. s19 00:01:50.640 --> 00:01:57.920 A second agent can check every response before it reaches the users. s20 00:01:57.920 --> 00:02:08.800 And four, neuro-symbolic guardians, you rule life in Python, not in the prompt, and the model cannot escape them. s21 00:02:08.800 --> 00:02:18.920 So, five, runtime guardians, because if you don't want to block, you can steer, and you don't you don't need to block everything. s22 00:02:18.920 --> 00:02:26.200 And when a rule fires, the agent self-correct and complete the task. s23 00:02:26.200 --> 00:02:29.320 No hard stop, no user advice. s24 00:02:29.320 --> 00:02:38.400 So, for each technique, I will show you the agent without it, and then with it, so you can compare. s25 00:02:38.400 --> 00:02:45.640 And all the demos uh I'm using a travel agent that I built using a Strands agent. s26 00:02:45.640 --> 00:02:54.040 And a Strands agent is a open-source agent framework that we maintain on AWS. s27 00:02:56.120 --> 00:02:59.519 And I am Elizabeth Fuentes Leon. s28 00:02:59.519 --> 00:03:02.560 I am a developer advocate for AWS. s29 00:03:02.560 --> 00:03:05.600 I'm focused on agentic application. s30 00:03:05.600 --> 00:03:19.160 And here, in this QR code, you will find everything that you will need to recreate all these techniques that I'm going to show you in a moment. s31 00:03:21.720 --> 00:03:29.160 So, let's get into Let's get into it. s32 00:03:31.880 --> 00:03:39.040 So, semantic tool selection, our travel agent has 29 tools. s33 00:03:39.040 --> 00:03:56.960 Flights, hotels, payments, weather, cancellations, all all our dummy tools are not like a travel agent for real, but every time that user sends a message, all the 29 tools description s34 00:03:56.960 --> 00:04:00.200 go into the context windows. s35 00:04:00.520 --> 00:04:07.000 The model reads all of them before deciding what to do. s36 00:04:07.000 --> 00:04:20.280 And if your agent has memory, the rows to every conversation adds more context that gets sent with every single message. s37 00:04:20.280 --> 00:04:30.520 And you pay for every single one of those tokens, whether the model ends up using the tool or not. s38 00:04:30.520 --> 00:04:39.480 To understand where those tokens come from, you need to see what a tool actually looks like to the model. s39 00:04:39.480 --> 00:04:52.480 In a Strands, you write a function with the tool decorator that's the tool, and a name, a description, and a docstring typed parameters. s40 00:04:52.480 --> 00:05:06.560 Then, Strands take that and generate a schema with name, description, parameters, and the schema is what goes into the context windows on every call. s41 00:05:07.200 --> 00:05:17.400 Each tool schema is about 17 or 200 tokens, depending of how many parameters it has. s42 00:05:17.400 --> 00:05:18.988 If our travel agent s43 00:05:18.988 --> 00:05:19.240 [snorts] s44 00:05:19.240 --> 00:05:31.040 has 29 tools, that adds up to somewhere around 3,000 tokens per call, just for the tool description. s45 00:05:31.040 --> 00:05:38.040 Before your message, before the response, every single call. s46 00:05:38.040 --> 00:05:48.480 By creating a tool database, we can filter the tools that the agent may need before the agent the symbol. s47 00:05:48.480 --> 00:05:55.880 With this filter, the model sees only three most relevant tools. s48 00:05:55.880 --> 00:06:02.960 Tokens usage drops from thousands to fewer than 300. s49 00:06:02.960 --> 00:06:05.240 Let me show you in the code. s50 00:06:05.240 --> 00:06:12.080 Here in my cure ID, this is the Let me clear all the outputs. s51 00:06:12.080 --> 00:06:16.640 So, first we need to install the requirements. s52 00:06:16.640 --> 00:06:18.760 Here is the requirement file. s53 00:06:18.760 --> 00:06:27.800 This is a Jupyter notebook because it's more simple to show everything, but there is one application that you can run if it is more comfortable for you. s54 00:06:27.800 --> 00:06:29.200 So, here are the requirements. s55 00:06:29.200 --> 00:06:38.120 I have the A strands agent and because I'm using open AI as a model invocation, I'm using the API from open AI. s56 00:06:38.120 --> 00:06:41.840 And I need a strands agent for open AI. s57 00:06:41.840 --> 00:06:47.600 You can use open AI strands agent with all our my god, almost all the model provider. s58 00:06:47.600 --> 00:06:55.840 And with a strand we Amazon better of course because we as AWS, we are the maintaining of this framework. s59 00:06:55.840 --> 00:07:02.680 And to use a Amazon better, you don't need to add the model provider. s60 00:07:02.680 --> 00:07:12.600 And because I need to embed in I need to create embeddings for my vector tools database, I'm using the sentence transformer. s61 00:07:12.600 --> 00:07:17.919 This is a super simple model that runs locally, so it's free. s62 00:07:17.919 --> 00:07:24.120 So, you want you can run this locally you computer without expense another model embedding. s63 00:07:24.120 --> 00:07:28.680 And you can use a strands with all llama too. s64 00:07:28.680 --> 00:07:38.280 So, if you have a local model, you can run everything in this in your computer for free without expense any tokens. s65 00:07:38.280 --> 00:07:45.440 And I use it files as my vector store, local, super simple. s66 00:07:45.440 --> 00:07:47.920 And well, right now I'm going to use a Neo4j. s67 00:07:47.920 --> 00:07:52.520 This is for other demo that I'm going to show you in a moment. s68 00:07:52.520 --> 00:07:57.920 And because I'm using some environments, I'm using Python of And let's see. s69 00:07:57.920 --> 00:07:59.840 So here I want to run this. s70 00:07:59.840 --> 00:08:01.080 Let me put it bigger. s71 00:08:01.080 --> 00:08:03.400 I know that you have it some problems. s72 00:08:03.400 --> 00:08:04.360 Bigger. s73 00:08:04.360 --> 00:08:06.880 Little more little less. s74 00:08:06.880 --> 00:08:10.080 And here, this is better. s75 00:08:10.080 --> 00:08:11.440 So. s76 00:08:11.440 --> 00:08:12.880 I style my requirement. s77 00:08:12.880 --> 00:08:14.000 I already did that. s78 00:08:14.000 --> 00:08:18.280 And I'm using OpenAI, so I need my my API. s79 00:08:18.280 --> 00:08:19.560 My API key. s80 00:08:19.560 --> 00:08:21.000 API key. s81 00:08:21.000 --> 00:08:23.600 And here, I invoke my strands. s82 00:08:23.600 --> 00:08:27.840 I need the agent because I want to be like agent that I'm using OpenAI. s83 00:08:27.840 --> 00:08:31.320 And I have a bunch of tool dummy tools here. s84 00:08:31.320 --> 00:08:33.280 I let I going to show you in a moment. s85 00:08:33.280 --> 00:08:44.400 And I want to use this because I have some um functions that I'm using to uh create the build the index for my vector store. s86 00:08:44.400 --> 00:08:49.839 And I need the search tools when I'm using the vector store, I put my query there. s87 00:08:49.839 --> 00:08:57.640 I search for the I search for the tool that I'm going to use using vector um um search for vectors. s88 00:08:57.640 --> 00:08:59.360 And I swap the tools. s89 00:08:59.360 --> 00:09:01.240 I going to show you that in a moment. s90 00:09:01.240 --> 00:09:03.800 So let me show you all my dummies tools. s91 00:09:03.800 --> 00:09:08.120 Where is the engage register here. s92 00:09:08.320 --> 00:09:10.440 Here I have all my tools. s93 00:09:10.440 --> 00:09:12.160 Uh this is for swap. s94 00:09:12.160 --> 00:09:14.040 Search, build index. s95 00:09:14.040 --> 00:09:16.040 And where are my tools? s96 00:09:16.040 --> 00:09:18.000 Shoo shoo shoo shoo shoo here. s97 00:09:18.000 --> 00:09:21.480 So here are all the dummy tools I created for this. s98 00:09:21.480 --> 00:09:22.760 You know, it's super. s99 00:09:22.760 --> 00:09:24.520 This is a demo, please. s100 00:09:24.520 --> 00:09:28.200 It's not something that you can use to go to production. s101 00:09:28.200 --> 00:09:29.600 No, please. s102 00:09:29.600 --> 00:09:30.720 And okay. s103 00:09:30.720 --> 00:09:32.400 So where is this this one? s104 00:09:32.400 --> 00:09:34.720 So I run this. s105 00:09:34.720 --> 00:09:36.920 And I build my semantic index. s106 00:09:36.920 --> 00:09:39.400 I already did that via Yeah. s107 00:09:39.400 --> 00:09:49.160 So, I have 29 tools here, 29 dummy tools, and I want to test this with a lot of different queries, and this is the ground truth. s108 00:09:49.160 --> 00:10:00.560 I have a ground truth because I know what is the best um the best tool to answer the question, so we can know if our agent is okay or not okay. s109 00:10:00.560 --> 00:10:02.360 So, let's run this. s110 00:10:02.360 --> 00:10:06.520 We have 90 queries and 29 tools. s111 00:10:06.520 --> 00:10:09.480 So, this is some helper functions. s112 00:10:09.480 --> 00:10:10.840 I don't care. s113 00:10:10.840 --> 00:10:13.280 Yeah, I care, but I don't want to explain you that. s114 00:10:13.280 --> 00:10:20.960 So, this is my agent, my traditional agent, where I'm putting all my 29 tools inside the agent. s115 00:10:20.960 --> 00:10:24.000 I have my model, some helper function. s116 00:10:24.000 --> 00:10:25.360 This is my agent. s117 00:10:25.360 --> 00:10:29.720 I only This is the way I create a agent using a strange agent. s118 00:10:29.720 --> 00:10:34.680 I put all my tools, and my system prompt is you are a travel assistant. s119 00:10:34.680 --> 00:10:36.520 You select correct tool to answer question. s120 00:10:36.520 --> 00:10:38.840 Oh my god, super. s121 00:10:38.840 --> 00:10:40.440 And the model. s122 00:10:40.440 --> 00:10:41.800 Let's run this. s123 00:10:41.800 --> 00:10:43.760 And this is whatever. s124 00:10:43.760 --> 00:10:51.440 This is some function to count the tokens because in inside this strange agent, you can you can count the tokens. s125 00:10:51.440 --> 00:10:57.720 You You can know how many tokens are using for in and out the the agent. s126 00:10:57.720 --> 00:11:00.680 This is some helper function, and yeah. s127 00:11:00.680 --> 00:11:09.360 This is Every time that I put a uh a prompt inside this agent, I'm only sending the prompt, and the agent respond. s128 00:11:09.360 --> 00:11:13.200 Then, I I use a new agent because it's a a fork. s129 00:11:13.200 --> 00:11:15.600 So, I'm not having a conversation with the agent. s130 00:11:15.600 --> 00:11:20.000 I only send a uh a question, and I receiving an answer. s131 00:11:20.000 --> 00:11:25.080 So, for each question, it is spend around 2,000 tokens. s132 00:11:25.080 --> 00:11:30.080 So, in and not always it give me the right answer. s133 00:11:30.080 --> 00:11:31.000 Here. s134 00:11:31.000 --> 00:11:40.240 So, yes, I have um I'm not so good at accuracy, and the average is 1 1,000 tokens. s135 00:11:40.240 --> 00:11:44.720 Now, let's use my new AI with the semantic approach. s136 00:11:44.720 --> 00:11:56.080 The thing The first thing that I do is for every single query, for every single prompt, I send the query to my search tools, and I going to receive s137 00:11:56.080 --> 00:12:08.880 the top two the top K three most relevant with more probability to response my query because this is a semantic search inside my vector store. s138 00:12:08.880 --> 00:12:12.280 And then I use that response. s139 00:12:12.280 --> 00:12:18.080 I going to select that name, and I going to put that tools inside my AI. s140 00:12:18.080 --> 00:12:28.240 So, I only going to use the three tools that this semantic search retrieve me for my AI, and I send that only tools. s141 00:12:28.240 --> 00:12:31.800 And this is my function helper, and that's it. s142 00:12:31.800 --> 00:12:35.960 So, I going to send the all the queries again the same. s143 00:12:35.960 --> 00:12:41.760 And for the first question, I have 4,000 tokens three blah blah blah. s144 00:12:41.760 --> 00:12:51.640 And yeah, we have a huge different because I don't send the all the 29 to tools in all the queries. s145 00:12:51.640 --> 00:12:53.960 So, yeah, bam bam. s146 00:12:53.960 --> 00:12:57.520 And okay, semantic memory. s147 00:12:57.520 --> 00:12:59.280 Yeah, okay, now come on. s148 00:12:59.280 --> 00:13:00.080 Finish. s149 00:13:00.080 --> 00:13:03.040 So, let's let's let's go to the next one. s150 00:13:03.040 --> 00:13:08.480 So, this engine is only sending a question, and I receive a response. s151 00:13:08.480 --> 00:13:11.360 I don't have any conversation with this engine. s152 00:13:11.360 --> 00:13:14.280 What happen if I start to have a conversations? s153 00:13:14.280 --> 00:13:16.440 If the engine remember me? s154 00:13:16.440 --> 00:13:22.080 So, we need to we need to send only the tools that this engine is going to use. s155 00:13:22.080 --> 00:13:32.640 Because if I give the all the tools I'm using in the conversation history, uh then I it's going to be in a moment that I want to have the 29 tools inside my engine. s156 00:13:32.640 --> 00:13:44.040 So, we are not resolving the we are not having a a um We are not resolving our problems in the engine when we having a conversation with the engine. s157 00:13:44.040 --> 00:13:49.120 So, we need to put the tools and then we need to remove the tools. s158 00:13:49.120 --> 00:13:53.960 And we can do that with the swap tools. s159 00:13:53.960 --> 00:13:55.440 And let's run this. s160 00:13:55.440 --> 00:14:06.760 And what swap tools I got run this, no. Let me Let me show you first the swap tools functions here the register. s161 00:14:06.760 --> 00:14:23.080 I delete this because with a strands in each invocation, you have complete control of the status because this is a a gentle loop and you can we use this a gentle loop s162 00:14:23.080 --> 00:14:33.680 how for because this is a gentle loop and you can do almost whatever you want. s163 00:14:33.680 --> 00:14:40.400 You can put and remove everything inside the loop with some few lines of code. s164 00:14:40.400 --> 00:14:41.920 So, let's see. s165 00:14:41.920 --> 00:14:43.839 Here we have the engine. s166 00:14:43.839 --> 00:14:47.720 We have the tool register through your through tool registers. s167 00:14:47.720 --> 00:14:51.160 It's something inside the engine status. s168 00:14:51.160 --> 00:14:55.520 So, we can clear the tools and that's it. s169 00:14:55.520 --> 00:15:03.160 So, in the next invocation, we can clear the all tools and we can add the new tools. s170 00:15:03.160 --> 00:15:07.560 Here, where is my I have a lot of things here. s171 00:15:09.440 --> 00:15:13.040 So, I run this. s172 00:15:16.320 --> 00:15:19.040 It's going to take a moment. s173 00:15:28.880 --> 00:15:35.600 And yes, we can see that in each invocation, my amount of talking is increasing. s174 00:15:35.600 --> 00:15:37.200 Why I have more? s175 00:15:37.200 --> 00:15:40.960 Because I'm sending the chat history as well. s176 00:15:40.960 --> 00:15:46.600 So, I send the tools that I need, only the tools that I need, and the chat history. s177 00:15:46.600 --> 00:15:54.440 So, that's why I can see that my amount of token is bigger and bigger. s178 00:15:59.600 --> 00:16:03.040 And the see, the accuracy is better. s179 00:16:03.040 --> 00:16:06.720 And yeah, we have more tokens. s180 00:16:08.800 --> 00:16:23.280 So, yes, probably we don't have the best accuracy here because this is a super demo with super dummies tools, and some queries are ambiguous on purpose because search for something, s181 00:16:23.280 --> 00:16:25.640 for something, check something. s182 00:16:25.640 --> 00:16:32.839 And the demo has a generic tools, dummy tools, with some similar name. s183 00:16:32.839 --> 00:16:40.800 And when all the 29 tools are visible, the model sometimes pick the wrong one. s184 00:16:40.800 --> 00:16:48.000 And with the filtering, those generic tools uh only appears in the query actually match them. s185 00:16:48.000 --> 00:16:52.320 So, you can run everything and you can test this. s186 00:16:52.320 --> 00:16:55.960 So, this is only that run only locally. s187 00:16:55.960 --> 00:17:00.880 But, what happen when you want to put this demo in production? s188 00:17:00.880 --> 00:17:05.000 You can of course, you can build a bigger, better store. s189 00:17:05.000 --> 00:17:07.040 I don't know, you can use Postgres. s190 00:17:07.040 --> 00:17:08.640 I think it's too much. s191 00:17:08.640 --> 00:17:14.839 But, we on AWS, we have Amazon Bedrock Agent Core. s192 00:17:14.839 --> 00:17:22.240 So, Amazon Bedrock Agent Core is a service dedicated only to agents in production. s193 00:17:22.240 --> 00:17:27.280 And in size of Agent Core, we have Agent Core Gateway. s194 00:17:27.280 --> 00:17:36.240 So, Agent Core Gateway Gateway allows you to build this index [clears throat] uh alone. s195 00:17:36.240 --> 00:17:40.040 So, you only have to say, "Hey, this is my tools." s196 00:17:40.040 --> 00:17:43.880 And Agent Core Gateway is going to build everything for you. s197 00:17:43.880 --> 00:17:48.120 And it can have the vector search inside. s198 00:17:48.120 --> 00:17:54.280 So, the routing layer is inside the Agent Core and it handles the tool selection automatically. s199 00:17:54.280 --> 00:18:01.080 So, you register your tools once and it find the right one for each request. s200 00:18:01.080 --> 00:18:06.000 It's the same principle but without infrastructure to manage. s201 00:18:06.000 --> 00:18:09.000 Now, let's see the next one. s202 00:18:10.960 --> 00:18:12.440 GraphRAG. s203 00:18:12.440 --> 00:18:16.760 You know, RAG is retrieval augmented generation. s204 00:18:16.760 --> 00:18:20.840 Is how agents access your own data. s205 00:18:20.840 --> 00:18:31.480 You take the user question, search your documents for the most similar content using vector search and pass what you find to the model. s206 00:18:31.480 --> 00:18:37.000 The model answer from that, it's work well for open question. s207 00:18:37.000 --> 00:18:39.520 Find me something about this topic. s208 00:18:39.520 --> 00:18:44.680 But there is a category of question where that breaks down. s209 00:18:44.680 --> 00:18:48.480 What is the average rating across all hotels in Paris? s210 00:18:48.480 --> 00:18:51.880 How many hotels have a pool? s211 00:18:51.880 --> 00:18:59.080 Vector search always returns something even when nothing is truly relevant. s212 00:18:59.080 --> 00:19:06.440 And the agent only sees the top end chunks of your all data at a time. s213 00:19:06.440 --> 00:19:12.560 It cannot aggregate, count, or traverse relationship across all the full data set. s214 00:19:12.560 --> 00:19:15.040 So, it estimates. s215 00:19:15.040 --> 00:19:22.040 And it present that estimates as a real fact, you know, a real answer. s216 00:19:22.040 --> 00:19:34.600 So, here we have graph you build a vector store and it retrieves three chunks from 300 documents and the model guesses. s217 00:19:34.600 --> 00:19:43.960 If we are using graph rack, you can run a query across all the and returns a compute results. s218 00:19:43.960 --> 00:19:55.960 Graph address this differently because instead of a retrieving text chunk, you can build a knowledge graph from the documents. s219 00:19:55.960 --> 00:20:00.000 Knows relationship and structured data. s220 00:20:00.000 --> 00:20:10.800 For the demo, I'm going to use Neo4j locally and the model is going to write a Cypher query to search it. s221 00:20:10.800 --> 00:20:14.480 Cypher query is Neo4j query language. s222 00:20:14.480 --> 00:20:16.640 It's similar to SQL. s223 00:20:16.640 --> 00:20:20.840 So, the graph run that query across all the data. s224 00:20:20.840 --> 00:20:27.760 And the model gets back a compute verified results, not a sample. s225 00:20:27.760 --> 00:20:32.760 And before running this demo, I'm going to start the dependents. s226 00:20:32.760 --> 00:20:33.720 Let me show you. s227 00:20:33.720 --> 00:20:36.800 Let me Let's go to the code. s228 00:20:37.960 --> 00:20:42.240 So, graph This is the notebook I'm going to share with you. s229 00:20:42.240 --> 00:20:45.240 So, we need to install the requirements again. s230 00:20:45.240 --> 00:20:48.200 What we have here, let me see. s231 00:20:48.200 --> 00:20:50.400 This is the first one, yeah. s232 00:20:50.400 --> 00:20:51.840 Requirements. s233 00:20:51.840 --> 00:20:55.120 I have I'm going to use OpenAI again. s234 00:20:55.120 --> 00:20:56.640 Let me just close this. s235 00:20:56.640 --> 00:21:01.440 I'm going to use Neo4j and we need Neo4j for graph rack. s236 00:21:01.440 --> 00:21:15.560 And for the engine that is using normal rack, we are going to be a super simple uh vector store in files and we are going to use again the sentence transformer. s237 00:21:15.560 --> 00:21:16.360 And let's see. s238 00:21:16.360 --> 00:21:20.760 I already have all right, I invoke [clears throat] my open AI. s239 00:21:21.000 --> 00:21:22.560 This is my tools. s240 00:21:22.560 --> 00:21:30.120 I I because I want to create some tools here, I want to use open AI and I use in graph database. s241 00:21:30.120 --> 00:21:34.120 And this is my I have my Neo4j locally. s242 00:21:34.120 --> 00:21:37.120 And let's run this. s243 00:21:37.120 --> 00:21:39.560 And this is something to check Neo4j. s244 00:21:39.560 --> 00:21:43.400 So, I'm building my uh vector store. s245 00:21:43.400 --> 00:21:47.120 This is my files and this is my Neo4j that I have here locally. s246 00:21:47.120 --> 00:21:48.400 I don't know if I can show you. s247 00:21:48.400 --> 00:21:49.560 I think not. s248 00:21:49.560 --> 00:21:54.600 And I have a tool, a normal tool created with a decorator. s249 00:21:54.600 --> 00:21:58.600 This is to search inside the my vector store. s250 00:21:58.600 --> 00:22:00.800 You see that's super I I have the query. s251 00:22:00.800 --> 00:22:10.080 I I create I I create the the embedding for my query and then I search inside the vector store and I have the query for the knowledge graph. s252 00:22:10.080 --> 00:22:20.840 So, here the model need to understand that to search in the query in the graph knowledge base, it need to build a Cypher query. s253 00:22:20.840 --> 00:22:25.800 So, I put that in the context because we already know how to build uh tools, right? s254 00:22:25.800 --> 00:22:27.640 I put that in the context. s255 00:22:27.640 --> 00:22:37.800 I have my driver to send the data to uh to connect with the vector store and to send the Cypher query that the model is going to create for me. s256 00:22:37.800 --> 00:22:41.400 And this is to read the results and that's it. s257 00:22:41.400 --> 00:22:44.480 That's the only tool that I need to search in my knowledge graph. s258 00:22:44.480 --> 00:22:48.080 Okay, this is my model open AI. s259 00:22:48.080 --> 00:22:50.160 This is my rag. s260 00:22:50.160 --> 00:22:52.800 My rag engine and uh this is my graph engine. s261 00:22:52.800 --> 00:22:56.080 I have two different engine to compare the results. s262 00:22:56.080 --> 00:22:57.560 Let's see. s263 00:22:57.560 --> 00:22:59.160 Well, I have a problem. s264 00:22:59.160 --> 00:23:00.840 Yeah, I'm ready. s265 00:23:00.840 --> 00:23:04.360 So, let's do this round the first test. s266 00:23:04.360 --> 00:23:06.360 Aggregation. s267 00:23:10.880 --> 00:23:13.040 Here, what is the question? s268 00:23:13.040 --> 00:23:17.200 What is the average guest rating across all hotels in Paris? s269 00:23:17.200 --> 00:23:21.200 So, meanwhile this engine is Oh, it's already done. s270 00:23:21.200 --> 00:23:31.360 So, what I have here I have the average guest rating across the hotels listed in Paris is Um it calculate, you know. s271 00:23:31.360 --> 00:23:41.600 When something When I'm using rag, it go to the to the vector store, it receive the n uh possible answers for this question. s272 00:23:41.600 --> 00:23:50.800 Because this is an aggregation, it's going to use the data that it receive to build a mathematical uh operation. s273 00:23:50.800 --> 00:23:52.200 This is the thing that I have here. s274 00:23:52.200 --> 00:23:56.760 So, everything that we see here is the is the model reasoning. s275 00:23:56.760 --> 00:24:06.760 So, when I run the graph engine on the mean, the engine is only going to give me the answer, you know, only these tokens here. s276 00:24:06.760 --> 00:24:12.280 Because the Cypher query, it can give me the mathematical operation itself. s277 00:24:12.280 --> 00:24:13.080 It can do that. s278 00:24:13.080 --> 00:24:22.960 So, we don't need the engines, the LLMs, the model uh do that for me because the Cypher query already give me the right question. s279 00:24:22.960 --> 00:24:26.920 So, it is 47 and the average. s280 00:24:26.920 --> 00:24:35.440 So, here in this uh first we are lucky because probably they are Yeah, only two hotels. s281 00:24:35.440 --> 00:24:44.520 But, what happened if this vector store have more than two or three hotels in the in the vector store? s282 00:24:44.520 --> 00:24:47.240 So, we are not going to give a real answer. s283 00:24:47.240 --> 00:24:54.720 It's going to the LLM is going to calculate with the only three answer that it receive. s284 00:24:54.720 --> 00:24:58.960 So, it's going to build this operation with only three hotels. s285 00:24:58.960 --> 00:25:05.560 But, if the vector store have more than three, we are going to have some problems. s286 00:25:05.560 --> 00:25:09.000 Something that I am you know uh yeah. s287 00:25:09.000 --> 00:25:11.000 So, let's run the next one. s288 00:25:11.000 --> 00:25:13.280 The prices counting. s289 00:25:13.280 --> 00:25:14.880 Something similar too. s290 00:25:14.880 --> 00:25:19.680 How many hotel have a swimming pools as a amenity? s291 00:25:19.680 --> 00:25:21.240 How many again? s292 00:25:21.240 --> 00:25:30.680 So, it give me the the traditional It's appear that the search did not return some specific information about hotels in Paris. s293 00:25:30.680 --> 00:25:31.720 Okay. s294 00:25:31.720 --> 00:25:33.760 What happened with the other one? s295 00:25:33.760 --> 00:25:37.560 There are currently no hotel that offer swimming pools. s296 00:25:37.560 --> 00:25:40.280 No I. There is no hotels. s297 00:25:40.280 --> 00:25:46.720 So, the other one is like mm would you like to ask about hotel with other specific amenity or information? s298 00:25:46.720 --> 00:25:50.960 It's like mm maybe it is or maybe I don't know. s299 00:25:50.960 --> 00:25:54.440 Yeah, it's not so accurate, okay? s300 00:25:54.440 --> 00:25:57.120 So, mostly conversation. s301 00:25:58.720 --> 00:26:00.560 Oh, what is my question? s302 00:26:00.560 --> 00:26:05.880 What are the room types and price for the highest rating hotels? s303 00:26:05.880 --> 00:26:08.840 What are the rooms types and price? s304 00:26:08.840 --> 00:26:10.680 This two question. s305 00:26:10.680 --> 00:26:14.400 So, it search the fact I had had only one. s306 00:26:14.400 --> 00:26:17.760 The highly rated hotels in any company Paris blah blah blah. s307 00:26:17.760 --> 00:26:19.880 I I don't speak French. s308 00:26:19.880 --> 00:26:22.600 With the guest rating is that one. s309 00:26:22.600 --> 00:26:26.000 However, I currently don't have Okay, blah blah blah. s310 00:26:26.000 --> 00:26:27.440 A lot of data there. s311 00:26:27.440 --> 00:26:30.280 So, what happened with the other one? s312 00:26:30.280 --> 00:26:34.920 Receiving notification for Oh, I have a error. s313 00:26:35.600 --> 00:26:38.480 I hear in the second one I receive the the question. s314 00:26:38.480 --> 00:26:40.880 The highly rated hotel is harmony blah blah blah. s315 00:26:40.880 --> 00:26:42.680 They offer following types. s316 00:26:42.680 --> 00:26:45.880 But, unfortunately these rooms are not available. s317 00:26:45.880 --> 00:26:48.880 So, I receiving a answer. s318 00:26:48.880 --> 00:26:50.920 I don't receive a lot of blah blah blah there. s319 00:26:50.920 --> 00:26:54.560 I only receive the answer what I need. s320 00:26:54.560 --> 00:26:58.720 And for the next one, how was the meditation? s321 00:26:59.640 --> 00:27:00.680 What is the question? s322 00:27:00.680 --> 00:27:03.280 Tell me about hotels in Antarctica. s323 00:27:03.280 --> 00:27:06.280 A spoiler, there is no hotel in Antarctica. s324 00:27:06.280 --> 00:27:10.760 There is zero hotels in Antarctica and let's see. s325 00:27:10.760 --> 00:27:22.600 It's appear that the certain order to specific information about hotel Okay, because they are I search for the other have the tail blah blah blah in the If you are looking for particular that experience s326 00:27:22.600 --> 00:27:25.440 specifically inquire about visiting Antarctica. s327 00:27:25.440 --> 00:27:29.160 No. Please let me know and I can assist you for it. s328 00:27:29.160 --> 00:27:34.840 Okay, a lot of tokens that I spending there in that answer for Elena. s329 00:27:34.840 --> 00:27:36.280 What happened with the other one? s330 00:27:36.280 --> 00:27:39.080 They are currently no hotel listed in Antarctica. s331 00:27:39.080 --> 00:27:45.000 Of course, because it create a cyber query, the cyber query and it receive zero. s332 00:27:45.000 --> 00:27:49.480 So, it no, it give me a honest answer. s333 00:27:49.480 --> 00:27:55.000 Here's summary that close grade for me for this Neo4j notebook. s334 00:27:55.000 --> 00:28:00.800 But here is something that I want to show you before go to the other one. s335 00:28:00.800 --> 00:28:07.320 So, here something that I love for Neo4j because why I'm using Neo4j? s336 00:28:07.320 --> 00:28:16.760 Because in the library the Neo4j give me uh it can build it use a LLM. s337 00:28:16.760 --> 00:28:21.400 I'm using the OpenAI as well to build the knowledge graph. s338 00:28:21.400 --> 00:28:24.200 So, how I build my knowledge graph? s339 00:28:24.200 --> 00:28:36.000 I only have a bunch of data, a bunch of txt only text and I send that data to the Neo4j library that I miss out. s340 00:28:36.000 --> 00:28:51.000 Okay, I send that data to this and Neo4j using this all this um library just right here, it can understand all my data and it can build the graph for me. s341 00:28:51.000 --> 00:29:00.760 So, I don't need to create that using the simple knowledge graph pipeline inside the knowledge graph library. s342 00:29:00.760 --> 00:29:03.760 So, that's why I'm using Neo4j. s343 00:29:03.760 --> 00:29:04.760 It's amazing. s344 00:29:04.760 --> 00:29:06.400 It's super simple to use. s345 00:29:06.400 --> 00:29:08.240 So, I invite you to use it at that. s346 00:29:08.240 --> 00:29:12.840 So, let's go to the next one. s347 00:29:13.720 --> 00:29:17.280 So, multi-agent validation. s348 00:29:17.360 --> 00:29:21.200 Sometimes an agent fails, and nobody find out. s349 00:29:21.200 --> 00:29:28.520 It calls a tool, and the tool returns an error, and the agent does not surface that error. s350 00:29:28.520 --> 00:29:32.800 It generate a confidence success response instead. s351 00:29:32.800 --> 00:29:34.760 The user thinks it work. s352 00:29:34.760 --> 00:29:36.520 You think it work. s353 00:29:36.520 --> 00:29:37.840 It didn't not. s354 00:29:37.840 --> 00:29:43.320 The agent acts and validate its own output in the same loop. s355 00:29:43.320 --> 00:29:47.320 There's no separation, no second opinion. s356 00:29:47.320 --> 00:29:53.920 So, when something goes wrong, it rationalizes and tells you it's okay. s357 00:29:53.920 --> 00:29:55.240 It work. s358 00:29:55.240 --> 00:29:57.200 Here is what happens. s359 00:29:57.200 --> 00:30:10.040 Inside a single agent, when it fail, it calls the tool, gets an error, rationalizes it, and returns a success response. s360 00:30:10.040 --> 00:30:13.800 The user never sees the error. s361 00:30:13.800 --> 00:30:19.360 You can address this by adding a validation layer. s362 00:30:19.360 --> 00:30:24.040 You can have three agents in sequence. s363 00:30:24.040 --> 00:30:30.400 One acts, one checks, and other approve or rejects. s364 00:30:30.400 --> 00:30:36.720 Strands agent has a built-in class for this called swarm. s365 00:30:36.720 --> 00:30:42.080 It manage the handoff between agents automatically. s366 00:30:42.080 --> 00:30:48.520 You just the role of each agent in a system prompt. s367 00:30:48.520 --> 00:30:52.120 And let me show you that. s368 00:30:53.760 --> 00:30:58.080 So, this demo only needs a Swarms agent with open AI integration. s369 00:30:58.080 --> 00:31:01.240 Let's see the requirements here. s370 00:31:01.240 --> 00:31:03.040 All you need of is Swarms agent. s371 00:31:03.040 --> 00:31:04.880 We don't need anything more. s372 00:31:04.880 --> 00:31:08.520 Let's go to the notebook here. s373 00:31:08.520 --> 00:31:16.120 And the key important thing here is the already running is Let me see here. s374 00:31:16.120 --> 00:31:18.160 Is the Swarm. s375 00:31:18.160 --> 00:31:30.720 The Swarm it's what lets you to connect multiple agents together without creating like a for a while manually to put the all these agents together. s376 00:31:30.720 --> 00:31:37.160 It can build a chain and manage the handoff between them automatically. s377 00:31:37.160 --> 00:31:40.880 So, here we are going to create three agents. s378 00:31:40.880 --> 00:31:47.040 Let me go down because we have the normal and we have some ground truth data. s379 00:31:47.040 --> 00:31:49.840 So, here is to create a a single agent. s380 00:31:49.840 --> 00:31:54.520 So, we have we are going to test three different scenarios to validate a booking. s381 00:31:54.520 --> 00:31:58.000 This They we expect true to enable hotels. s382 00:31:58.000 --> 00:32:02.000 We expect false and non-existing hotels and missing booking. s383 00:32:02.000 --> 00:32:03.760 We expect false, too. s384 00:32:03.760 --> 00:32:06.400 This is how we created single agent. s385 00:32:06.400 --> 00:32:08.840 We need a prompt, some tools. s386 00:32:08.840 --> 00:32:12.600 And we are using the tools that we have here. s387 00:32:12.600 --> 00:32:15.920 And here we know that this is the data. s388 00:32:15.920 --> 00:32:20.200 So, the agents can can give out the answer that we are looking for. s389 00:32:20.200 --> 00:32:21.400 And that's it. s390 00:32:21.400 --> 00:32:23.440 Let me go to the other ones. s391 00:32:23.440 --> 00:32:25.000 Oh, wait here. s392 00:32:25.000 --> 00:32:27.880 So, how we build the Swarm? s393 00:32:27.880 --> 00:32:29.080 I already run this. s394 00:32:29.080 --> 00:32:30.640 I want to show you this one. s395 00:32:30.640 --> 00:32:33.800 So, we are going to build three different agents. s396 00:32:33.800 --> 00:32:37.480 The executor, the validator and the critic. s397 00:32:37.480 --> 00:32:40.000 The we have a system prompt. s398 00:32:40.000 --> 00:32:43.712 You are executor agent for a hotel booking system. s399 00:32:43.712 --> 00:32:44.160 [clears throat] s400 00:32:44.160 --> 00:32:49.400 Use the provide tools to fulfill requests accurately and blah blah blah. s401 00:32:49.400 --> 00:32:52.240 The validator is you are a validator agent. s402 00:32:52.240 --> 00:32:58.200 Review what the executor did and output exactly on off. s403 00:32:58.200 --> 00:33:03.640 And we have the critics that is going to set here approve or no reject. s404 00:33:03.640 --> 00:33:05.520 Let's run this. s405 00:33:05.520 --> 00:33:06.720 And oh, what is here? s406 00:33:06.720 --> 00:33:08.120 Our model didn't define the model. s407 00:33:08.120 --> 00:33:09.800 Sorry. s408 00:33:09.800 --> 00:33:10.880 Sorry, please. s409 00:33:10.880 --> 00:33:15.480 Forgive me the life because I didn't run this one. s410 00:33:17.440 --> 00:33:21.600 Let's run now because this is the the normal. s411 00:33:21.600 --> 00:33:24.280 And the model is there, right? s412 00:33:24.280 --> 00:33:25.320 What? s413 00:33:25.320 --> 00:33:26.560 A little bias. s414 00:33:26.560 --> 00:33:28.840 Run that. s415 00:33:29.160 --> 00:33:30.640 What's happening with you? s416 00:33:30.640 --> 00:33:31.680 Let me go this. s417 00:33:31.680 --> 00:33:33.360 Let me copy and paste. s418 00:33:33.360 --> 00:33:34.720 This one. s419 00:33:34.720 --> 00:33:37.720 I don't know why this give me problem. s420 00:33:39.400 --> 00:33:41.880 Multi-agent swarm is the time the same name. s421 00:33:41.880 --> 00:33:43.080 Come on. s422 00:33:43.080 --> 00:33:44.880 Yeah, you can see this is life. s423 00:33:44.880 --> 00:33:47.720 I don't going to edit this. s424 00:33:47.840 --> 00:33:49.920 So, yes. s425 00:33:49.920 --> 00:33:53.360 Okay, we have Thank you. s426 00:33:53.360 --> 00:33:56.520 And the swarm, how we how we create the swarm? s427 00:33:56.520 --> 00:34:05.240 We have swarm and the swarm function, we put all the agents together and the entry point is the executor. s428 00:34:05.240 --> 00:34:10.560 So, all everything is going to start in the executor and then it's going to hand off. s429 00:34:10.560 --> 00:34:13.399 And six time because you can handle that, too. s430 00:34:13.399 --> 00:34:15.360 So, get the swarm final response. s431 00:34:15.360 --> 00:34:18.879 Let's do a for and they send all the response. s432 00:34:18.879 --> 00:34:22.240 And book the Grand Hotel. s433 00:34:22.280 --> 00:34:24.879 I have booked a Grand Hotel for Alice tonight. s434 00:34:24.879 --> 00:34:25.879 Booking blah blah blah. s435 00:34:25.879 --> 00:34:27.399 Hand off. s436 00:34:27.399 --> 00:34:28.520 Valid. s437 00:34:28.520 --> 00:34:29.720 Okay. s438 00:34:29.720 --> 00:34:30.720 Looks good. s439 00:34:30.720 --> 00:34:32.639 Critic, approve. s440 00:34:32.639 --> 00:34:36.440 So, the critics is approving the this. s441 00:34:36.440 --> 00:34:38.040 This is the other one. s442 00:34:38.040 --> 00:34:42.440 And we can run some comparison here if you want. s443 00:34:43.960 --> 00:34:53.080 So, here we can see that the swarm handles the flow between the between all the uh three different saying. s444 00:34:53.080 --> 00:35:00.680 It was what happens when the single agent tries to conflict something that does not exist in the system. s445 00:35:00.680 --> 00:35:13.680 And now the same request through the swarm, the executor gets the error, the validator catches, the critic rejects this, and the user never see a fabricator response. s446 00:35:13.680 --> 00:35:22.040 So, you can we can test here with the single agent that it's a suggested unknown entry and return success. s447 00:35:23.880 --> 00:35:30.280 And the swarm, it have a executor that go the error, validate saying, "Hey, come on, man. s448 00:35:30.280 --> 00:35:32.920 Hallucination and the critic they reject." s449 00:35:32.920 --> 00:35:36.120 So, the user sees a clear failure. s450 00:35:36.120 --> 00:35:41.040 The fabricator confirmation so missing the issue here. s451 00:35:41.040 --> 00:35:46.120 And you can run this to compare the two different agents. s452 00:35:46.120 --> 00:35:53.840 So, here we can see that the single agent it have everything like a yeah, it's okay, it's correct. s453 00:35:53.840 --> 00:36:00.080 But in the multi-agent swarm, you can see that the last one is a you know, missing issue here. s454 00:36:00.080 --> 00:36:02.120 Something is happening. s455 00:36:02.120 --> 00:36:05.720 Now, let's go to the next one. s456 00:36:05.960 --> 00:36:06.720 Okay. s457 00:36:06.720 --> 00:36:09.320 Neuro-symbolic Guardians. s458 00:36:09.320 --> 00:36:11.160 You have a rule for your agent. s459 00:36:11.160 --> 00:36:15.840 Let's uh say maximum 10 guesses per reservation. s460 00:36:15.840 --> 00:36:18.960 You write in in the system prompt. s461 00:36:18.960 --> 00:36:22.120 You even write in in the tool description. s462 00:36:22.120 --> 00:36:26.920 And the agent still calls the tool with 15. s463 00:36:26.920 --> 00:36:29.720 No, because it is ignoring you. s464 00:36:29.720 --> 00:36:34.560 Because prompts probably are suggestions, not constraints. s465 00:36:34.560 --> 00:36:38.040 The model process them as a text. s466 00:36:38.040 --> 00:36:42.160 Not as a logic it has to execute. s467 00:36:42.160 --> 00:36:44.120 It's probabilistic. s468 00:36:44.120 --> 00:36:47.160 Only code execute logic. s469 00:36:47.160 --> 00:36:52.640 A rule in the prompt the model read it as a suggestion. s470 00:36:52.640 --> 00:36:56.840 A rule in the code the model can not escape it. s471 00:36:56.840 --> 00:37:01.800 Neuro-symbolic guardians rules put the rules in the code. s472 00:37:01.800 --> 00:37:06.200 E-strands A and has a few that call him hooks. s473 00:37:06.200 --> 00:37:14.200 The the function that is trans called automatically at specific moments in the A and loop. s474 00:37:14.200 --> 00:37:18.440 It is this case right before a tool executes. s475 00:37:18.440 --> 00:37:20.160 So you have a hook. s476 00:37:20.160 --> 00:37:27.120 You write the rule, check the parameters and if they fails you cancel the call. s477 00:37:27.120 --> 00:37:31.040 Let me show you this in the code. s478 00:37:31.440 --> 00:37:34.760 Here we have the Jupiter notebook. s479 00:37:34.760 --> 00:37:37.520 We have the requirements here. s480 00:37:37.520 --> 00:37:42.800 Um what is the same requirements as the previous demos we only need I will install everything again. s481 00:37:42.800 --> 00:37:45.360 We only need E-strands and open AI. s482 00:37:45.360 --> 00:37:48.400 We don't need anything else. s483 00:37:48.400 --> 00:37:53.080 And um my API. s484 00:37:53.080 --> 00:37:56.080 And here is the thing important. s485 00:37:56.080 --> 00:38:02.200 In this demo are these three hooks provided in this trans base class. s486 00:38:02.200 --> 00:38:04.840 So you allows you to create the hook. s487 00:38:04.840 --> 00:38:10.040 So we have hook provider, hook register and before to call event. s488 00:38:10.040 --> 00:38:17.360 And the tool register is what E-strands pass you to register your call back. s489 00:38:17.360 --> 00:38:27.040 And before to call event is the event that fires every time the model is about to execute shadow. s490 00:38:27.040 --> 00:38:34.440 That last one is what makes it possible the intercept the call before it runs. s491 00:38:34.440 --> 00:38:42.320 So, you have the before two calls even and call we have after two calls even that we are not going to use that here. s492 00:38:42.320 --> 00:38:45.560 And let's see the rule in the prompt. s493 00:38:45.560 --> 00:38:52.200 Remember, the model reads as a text and it may follow or not follow them. s494 00:38:52.200 --> 00:38:53.440 So, let's see this. s495 00:38:53.440 --> 00:38:55.320 I want to run this. s496 00:38:55.320 --> 00:39:00.560 So, here we have some similar state that we are going to use in this engine. s497 00:39:00.560 --> 00:39:05.400 And let me this is the symbolic rule. s498 00:39:05.400 --> 00:39:12.400 So, booking rule if if rule booking rule as a this is something that we create. s499 00:39:12.400 --> 00:39:14.400 Let me show you the rules here. s500 00:39:14.400 --> 00:39:17.200 So, we have the rules. s501 00:39:17.200 --> 00:39:18.680 Rule one. s502 00:39:18.680 --> 00:39:20.240 Validate dates. s503 00:39:20.240 --> 00:39:22.640 Check in must be before check out. s504 00:39:22.640 --> 00:39:25.800 If what is validated they check out. s505 00:39:25.800 --> 00:39:27.360 This is something that it check. s506 00:39:27.360 --> 00:39:36.680 This is a something that is going to invoke when the rule is is uh is must have to use it. s507 00:39:36.680 --> 00:39:45.200 And uh to do We have another rule for max guesses. s508 00:39:45.200 --> 00:39:47.720 Maximum take guesses per booking. s509 00:39:47.720 --> 00:39:56.520 So, if I want to use I don't know I if I want to book in uh uh I want to do a booking for 11, it's going to block me. s510 00:39:56.520 --> 00:40:00.320 It's not going to reject the booking. s511 00:40:00.320 --> 00:40:02.640 And we have some confirmation rule. s512 00:40:02.640 --> 00:40:04.400 Payment before confirm. s513 00:40:04.400 --> 00:40:07.160 You know, you can confirm if you don't have the payment. s514 00:40:07.160 --> 00:40:08.920 Cancellation rule. s515 00:40:08.920 --> 00:40:14.600 Cancellation within the cannot cancel within 48 hours of check in. s516 00:40:14.600 --> 00:40:18.800 So, this is something that I've created here, you know, to this demo. s517 00:40:18.800 --> 00:40:21.520 So, I have booking rules and I have confirmation rules. s518 00:40:21.520 --> 00:40:24.280 I don't I use in here the cancellation rule. s519 00:40:24.280 --> 00:40:27.920 So, create validation hook. s520 00:40:27.920 --> 00:40:31.080 We create the validation hooks here, right? s521 00:40:31.080 --> 00:40:35.920 Like this, neuro-symbolic rule, and we use the hook provider. s522 00:40:35.920 --> 00:40:38.600 So, we have a bunch of code here. s523 00:40:38.600 --> 00:40:44.800 Then, we add the booking rules, the confirmation rules, and the state. s524 00:40:45.040 --> 00:40:50.960 And you can check this by yourself later, but we want to see the demo running. s525 00:40:50.960 --> 00:40:58.280 So, that we define the claim tools that we are going to use them for the booking hotels and the process payment. s526 00:40:58.480 --> 00:41:03.520 This is the the normal tools, you know, and we need to add that tools to the hooks. s527 00:41:03.520 --> 00:41:06.360 I think I add that. s528 00:41:07.520 --> 00:41:08.000 Yeah. s529 00:41:08.000 --> 00:41:10.080 Tool name, book hotel. s530 00:41:10.080 --> 00:41:21.160 So, this thing is going to get the eight when the book hotel is using the Yeah, the book hotel. s531 00:41:21.160 --> 00:41:22.200 Okay? s532 00:41:22.200 --> 00:41:25.480 So, let's create the agents for the comparison. s533 00:41:25.480 --> 00:41:31.000 So, we are going as the other demos, we have three different scenarios. s534 00:41:31.000 --> 00:41:33.160 Confirm booking without payment. s535 00:41:33.160 --> 00:41:36.080 Payment must be verified before the confirmation. s536 00:41:36.080 --> 00:41:38.680 This is the rule that I must trigger this. s537 00:41:38.680 --> 00:41:45.400 We have booking hotel exceeding guest limit, and we have valid booking for five guests. s538 00:41:46.040 --> 00:41:49.000 So, yeah, we have the three scenarios. s539 00:41:49.000 --> 00:41:54.120 And we have the normal engine, the normal engine, and the vaseline engine. s540 00:41:54.120 --> 00:41:58.400 And we have the engine with the neuro the neuro-symbolic guardians. s541 00:41:58.400 --> 00:42:02.800 So, we have the hook, the neuro-symbolic hook. s542 00:42:02.840 --> 00:42:04.400 Uh this is the same. s543 00:42:04.400 --> 00:42:05.760 Here, yeah. s544 00:42:05.760 --> 00:42:11.604 And we add this If you can see, the normal engine s545 00:42:11.604 --> 00:42:11.840 [snorts] s546 00:42:11.840 --> 00:42:12.960 is only three lines. s547 00:42:12.960 --> 00:42:19.240 The tools, the model, and this is the line that give us the difference between these two engines. s548 00:42:19.240 --> 00:42:21.240 So, I already run that. s549 00:42:21.240 --> 00:42:29.120 Now, let's go to confirm a booking without payment. s550 00:42:29.120 --> 00:42:31.080 So, I run this my engines. s551 00:42:31.080 --> 00:42:33.360 The first one is Abis. s552 00:42:33.360 --> 00:42:34.120 Come on, man. s553 00:42:34.120 --> 00:42:41.400 I Yeah, because I Yeah, it confirmed the booking without payment because it doesn't have any rules and the promise is super basic. s554 00:42:41.400 --> 00:42:49.640 And the other one is Let's open this and is booking blocked. s555 00:42:49.640 --> 00:42:52.960 Payment must be verified before the confirmation. s556 00:42:52.960 --> 00:42:53.960 Thank you. s557 00:42:53.960 --> 00:42:55.720 So, it's okay. s558 00:42:55.720 --> 00:42:57.520 Now, let's run the other one. s559 00:42:57.520 --> 00:43:00.360 Test second scenario. s560 00:43:01.880 --> 00:43:03.680 What is the question? s561 00:43:03.680 --> 00:43:06.160 Booking hotel access guest limit. s562 00:43:06.160 --> 00:43:20.240 So, the question is I Yeah, I have more than So, here the hotel Abis successful booking for Yeah, because the promise is basic and I don't put in any rule there. s563 00:43:20.240 --> 00:43:25.840 And for the other one, it seemed that the Grand Hotel has maximum capacity of 10 guests per booking. s564 00:43:25.840 --> 00:43:30.000 Additional booking must be made at last 1 day in advance. s565 00:43:30.000 --> 00:43:31.040 I don't remember the day. s566 00:43:31.040 --> 00:43:34.280 Probably This is something that they give them. s567 00:43:34.280 --> 00:43:39.640 And to validate booking is both engines execute all rules pass. s568 00:43:39.640 --> 00:43:42.360 Because they Yeah, it's only a validation for booking. s569 00:43:42.360 --> 00:43:45.000 So, book hotel for blah blah blah. s570 00:43:45.000 --> 00:43:47.200 All rules passes. s571 00:43:47.200 --> 00:43:49.920 It seemed that the booking is still needs to make. s572 00:43:49.920 --> 00:43:52.000 I don't I didn't remember the question. s573 00:43:52.000 --> 00:43:55.520 So, booking is made provide I Yeah. s574 00:43:55.520 --> 00:43:57.080 Okay. s575 00:43:57.080 --> 00:43:58.400 Okay, I get it. s576 00:43:58.400 --> 00:44:04.000 And Now, run all the thing this question. s577 00:44:04.000 --> 00:44:06.960 Confirming without payment, booking save max. s578 00:44:06.960 --> 00:44:07.960 And what happened here? s579 00:44:07.960 --> 00:44:10.120 This is all the scenarios that we just run. s580 00:44:10.120 --> 00:44:12.640 So, this is awesome thing to compare the results. s581 00:44:12.640 --> 00:44:19.520 So, Bali booking king secure five guesses, allowance, correct, wrong. s582 00:44:19.800 --> 00:44:23.760 50, blocked, wrong, correct. s583 00:44:23.760 --> 00:44:31.360 Confirm, so there we have here the comparison between the two ends. s584 00:44:33.880 --> 00:44:41.520 So, what we have what we have here is same model, same tools, same prompt. s585 00:44:41.520 --> 00:44:47.960 And the different that outcome because the rules are in Python knowing the prompt. s586 00:44:47.960 --> 00:45:01.680 This pattern of enforcing rules in code before the tools run is also what Amazon Aion core policies service that we have does at the structural level. s587 00:45:01.680 --> 00:45:06.760 So, the same concept but managed for you in production. s588 00:45:06.760 --> 00:45:08.880 And you only have to create the rules. s589 00:45:08.880 --> 00:45:12.440 So, the hooks are all or nothing. s590 00:45:12.440 --> 00:45:16.640 They block everything or approve. s591 00:45:16.640 --> 00:45:22.880 But sometimes you want the agent to adjust and keep going. s592 00:45:22.880 --> 00:45:26.520 No and stop and leave the user waiting. s593 00:45:26.520 --> 00:45:30.320 That is what I will show you in the next. s594 00:45:32.600 --> 00:45:38.720 Run time guardians, run time is still is still don't block. s595 00:45:38.720 --> 00:45:40.720 This is the next one. s596 00:45:40.720 --> 00:45:44.200 Hooks blocks unconditionally. s597 00:45:44.200 --> 00:45:48.960 The agent is stop and the user has to retry. s598 00:45:48.960 --> 00:45:53.680 For a hard constraint, that is exactly what you want. s599 00:45:53.680 --> 00:45:57.480 But sometimes the rule is soft. s600 00:45:57.480 --> 00:46:08.000 Maybe room fits for four guests, but a group of six could go book two different rooms. s601 00:46:08.000 --> 00:46:13.440 Or a flight is full, but there is ability on the next one. s602 00:46:13.440 --> 00:46:17.920 You do No, you don't want to block everything. s603 00:46:17.920 --> 00:46:23.560 Probably, you want the agent to find a option and complete the task. s604 00:46:23.560 --> 00:46:25.880 That is a steering. s605 00:46:25.880 --> 00:46:31.400 Here, we have the hook that fires and the task failed. s606 00:46:31.400 --> 00:46:37.240 And with agent control, it steered the models and the task complete. s607 00:46:37.240 --> 00:46:40.880 The other different here is operational. s608 00:46:40.880 --> 00:46:49.960 Because with the hooks, it's uh changing a rule means changing code and redeploy the whole harness, whole the engine. s609 00:46:49.960 --> 00:47:02.560 And with the agent control, which is the name of the open source library that we are going to use here, rules are registered on a local server via API. s610 00:47:02.560 --> 00:47:10.800 You open the agent without touching the agent code because the agent picks them up immediately. s611 00:47:10.800 --> 00:47:14.320 Let me show you that here in the code. s612 00:47:14.600 --> 00:47:17.520 Yeah, this is the notebook. s613 00:47:17.520 --> 00:47:21.360 In this demo, we only need a extra packet. s614 00:47:21.360 --> 00:47:26.560 The agent control is the key. s615 00:47:26.560 --> 00:47:32.680 So, the agent control is the one that helps us to create a steering. s616 00:47:32.680 --> 00:47:36.840 And here, we are using the setup control. s617 00:47:36.840 --> 00:47:46.200 This is a a little application, a little app that I create with the local server and the steering rules. s618 00:47:46.200 --> 00:47:48.160 Here, we have the local server. s619 00:47:48.160 --> 00:47:50.160 We have the control. s620 00:47:50.160 --> 00:47:50.243 They s621 00:47:50.243 --> 00:47:50.320 [snorts] s622 00:47:50.320 --> 00:47:51.640 are the steering rules. s623 00:47:51.640 --> 00:47:56.560 First, we have the the steer master guest. s624 00:47:56.560 --> 00:47:58.560 A steer, you know, guy. s625 00:47:58.560 --> 00:48:06.240 So, guy agent to reduce a guest count when exceeding maximum of 10. s626 00:48:06.240 --> 00:48:08.200 And he's going to steer. s627 00:48:08.200 --> 00:48:12.120 And we have some control that deny. s628 00:48:12.120 --> 00:48:14.560 For example, deny no payment. s629 00:48:14.560 --> 00:48:20.120 Block blocking confirmation without PR payment. s630 00:48:20.120 --> 00:48:26.760 And here, well, this is to to create the um the service. s631 00:48:26.760 --> 00:48:27.760 The service. s632 00:48:27.760 --> 00:48:32.280 And let's go to the uh notebook. s633 00:48:32.280 --> 00:48:33.840 So, I'm ready. s634 00:48:33.840 --> 00:48:36.320 Um Let me go here. s635 00:48:36.320 --> 00:48:36.680 Yeah. s636 00:48:36.680 --> 00:48:40.280 So, I need the environment. s637 00:48:40.360 --> 00:48:43.920 So, this is the agent. s638 00:48:43.920 --> 00:48:46.600 Um I'm going to have a error here. s639 00:48:46.600 --> 00:48:46.800 Wait. s640 00:48:46.800 --> 00:48:46.960 Wait. s641 00:48:46.960 --> 00:48:47.080 Wait. s642 00:48:47.080 --> 00:48:48.920 I'm going to use bedroom. s643 00:48:48.920 --> 00:48:49.160 Wait. s644 00:48:49.160 --> 00:48:50.360 Wait. s645 00:48:50.360 --> 00:48:55.960 No, then let me comment this one because this going to give me a error. s646 00:48:57.840 --> 00:49:00.680 And let's run this. s647 00:49:00.680 --> 00:49:04.600 Yeah, so we have the hooks as we did the demo. s648 00:49:04.600 --> 00:49:08.920 So, we have book any company Lisbon and we have some prompts. s649 00:49:08.920 --> 00:49:10.880 You are a hotel booking assistant. s650 00:49:10.880 --> 00:49:14.560 When booking, first describe what you will book in blah blah. s651 00:49:14.560 --> 00:49:17.560 And this is the prompt for my agent. s652 00:49:17.560 --> 00:49:18.480 Where is my agent? s653 00:49:18.480 --> 00:49:19.400 Agent control. s654 00:49:19.400 --> 00:49:23.680 This is for the Well, this is on some helper. s655 00:49:23.680 --> 00:49:27.720 And this is my hook that you already know because we create this. s656 00:49:27.720 --> 00:49:29.880 So, we have the system prompt. s657 00:49:29.880 --> 00:49:31.040 And we have the hooks. s658 00:49:31.040 --> 00:49:38.720 So, let's test this agent with the uh book any company Lisbon for 60 guests. s659 00:49:38.720 --> 00:49:46.280 And if you remember, this only can book for uh less than 10 guesses. s660 00:49:46.280 --> 00:49:49.200 And of course, he's blocking. s661 00:49:49.200 --> 00:49:54.240 Now, let's go to the AIN control new agent. s662 00:49:54.240 --> 00:50:02.200 Here we have two key importances or two key imports that are important for the AIN control SDK. s663 00:50:02.200 --> 00:50:11.000 So, we have the AIN control plugin that captures AIN events and sends them to the AIN control server. s664 00:50:11.000 --> 00:50:21.640 And we have the AIN control student handle that listen for a student decision for the server and delivers them back to the model. s665 00:50:21.640 --> 00:50:29.920 Together here they are what connects a student to the AIN control student logic. s666 00:50:29.920 --> 00:50:35.120 So, let's go this with this. s667 00:50:35.920 --> 00:50:45.920 So, the student agent is going to a river room for any company list board 50 guests. s668 00:50:45.920 --> 00:50:47.520 Let's see what it's doing. s669 00:50:47.520 --> 00:50:53.560 Yeah, I have to set for booking for a stay in the company list board for 50 guests for May. s670 00:50:53.560 --> 00:50:57.400 So, it's the reservation have been split into two rooms. s671 00:50:57.400 --> 00:51:00.480 So, it took that it in self. s672 00:51:00.480 --> 00:51:01.680 You know? s673 00:51:01.680 --> 00:51:07.440 One room for one and other room for five and that's it. s674 00:51:07.440 --> 00:51:11.400 So, use hook for hard constraints. s675 00:51:11.400 --> 00:51:13.295 AIN control for software s676 00:51:13.295 --> 00:51:13.800 [snorts] s677 00:51:13.800 --> 00:51:15.120 rule. s678 00:51:15.120 --> 00:51:20.480 Now, we have five technique or running locally. s679 00:51:20.480 --> 00:51:29.240 But, how do you take this to production without maintaining service, without building infrastructure? s680 00:51:29.280 --> 00:51:31.880 Let me show you how. s681 00:51:33.520 --> 00:51:39.880 Everything that I just built here in the previous demos runs locally. s682 00:51:39.880 --> 00:51:47.440 For the production version, Amazon Bedrock Bedrock Amazon Bedrock AIN core give you the run time. s683 00:51:47.440 --> 00:51:58.920 A gateway, a short term and a long term memory, a cloud watch observability, build in no servers to manage. s684 00:51:58.920 --> 00:52:03.440 Here the architecture, the strands engine runs inside the run time. s685 00:52:03.440 --> 00:52:10.480 And you know, you inside the run time, you can put every framework that you want, not only run a strands. s686 00:52:10.480 --> 00:52:16.840 The gateway rules tools calls to the lambda function automatically as a tools. s687 00:52:16.840 --> 00:52:22.960 And the steering rules from the previous demo live in the DynamoDB. s688 00:52:22.960 --> 00:52:29.680 So, you change them there and they are live on the next call. s689 00:52:29.680 --> 00:52:33.760 And you don't need to re-deploy anything. s690 00:52:33.760 --> 00:52:44.000 And you want to use Neo4j, of course, you can use Neo4j Aura DB, that is a external graph database, which also has a free tier. s691 00:52:44.000 --> 00:52:49.320 And the code is in the repo here. s692 00:52:51.480 --> 00:52:53.520 Is in the repo. s693 00:52:53.520 --> 00:53:00.240 And you will need AWS credentials if you're using Amazon Bella Betray in core. s694 00:53:00.240 --> 00:53:04.120 And in the resource link, there is some credits. s695 00:53:04.120 --> 00:53:15.680 I hope that you can find it because I always try to to give away some credit for AWS so you can deploy everything for free. s696 00:53:15.680 --> 00:53:27.040 And uh if you are new here in AWS, I have a repository that you can use to deploy everything over this architecture uh using a notebook, too. s697 00:53:27.040 --> 00:53:35.960 But, if you are familiar with CDK, cloud developer kit, you can do it that as well to deploy everything at once. s698 00:53:35.960 --> 00:53:38.920 Uh both options are in the repository, of course. s699 00:53:38.920 --> 00:53:45.000 And you can go deeper in Agent Core with all the documentation that I left there. s700 00:53:45.280 --> 00:53:52.120 So, please don't stop at the demo and try going to production with Amazon better agent car. s701 00:53:52.240 --> 00:53:56.040 So, let me bring it all back. s702 00:53:56.040 --> 00:53:59.920 Tokens waste on every request. s703 00:53:59.920 --> 00:54:03.760 You can fix it with semantic tool selection. s704 00:54:03.920 --> 00:54:07.960 And confident answer it's never computed. s705 00:54:07.960 --> 00:54:12.240 Possibly it's possible because you are asking how many. s706 00:54:12.240 --> 00:54:16.160 So, you can use graph rack and query the data. s707 00:54:16.160 --> 00:54:18.000 Don't sample it. s708 00:54:18.000 --> 00:54:21.880 Sometimes we have fabric case success confirmation. s709 00:54:21.880 --> 00:54:27.080 You can use multi-agent validation and second pass catch it. s710 00:54:27.080 --> 00:54:32.680 And rules the models uh quite a little skip it. s711 00:54:32.680 --> 00:54:39.040 You can use uh neurosymbolic guardians to enforce in the code. s712 00:54:39.040 --> 00:54:40.840 Don't trust in the prompt. s713 00:54:40.840 --> 00:54:50.280 And for the hard blocks that I stopped the user, you can use runtime SDR, self-correct, and finish. s714 00:54:51.160 --> 00:54:58.480 Each demo that I just showed you is in the in the repository as a notebook and an application as well. s715 00:54:58.480 --> 00:55:03.120 Start with the demo one and then go to the demo five. s716 00:55:03.120 --> 00:55:07.040 And if you want, deploy it in the productions. s717 00:55:07.040 --> 00:55:12.200 So, have you tried any of these in your own hands? s718 00:55:12.200 --> 00:55:17.880 Thank you to join me in this session and happy building.