Stop AI Agent Hallucinations: 5 Techniques + Production Patterns - Elizabeth Fuentes, AWS

AI Engineer · 55 min · 718 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:00Hi, today we are going to talk about how to stop AI agent hallucinations with five techniques beyond the prompt.
00:11Each one is a code change, not a prompt change.
00:16Let's see.
00:18Every time your AI agent responds, you are paying for the words going in and the words coming out.
00:26And your bill, you will see those calling tokens.
00:30Yeah.
00:31And the more tokens you are sending, the more you pay.
00:36And if what you send is not quite right, too much, or missing something important, your AI agent start to hallucinate.
00:46There are five techniques to help reduce tokens waste, improve accuracy, and catch failure before you submit them.
00:59And each one is a code change, not a prompt change at all.
01:05So, let's see it.
01:08First, we have semantic tool selection.
01:12You filter which tool go into context on every call.
01:18The model only see what it needs for that specific query.
01:23Second, we have graph rack.
01:26For precise queries like aggregation, counts, multi-hop reasoning, and you replace the text retrieval with a structured graph query.
01:38The model gets a compute a verifiable answer, not a sample as rack do.
01:47Three, multi-agent validation.
01:50A second agent can check every response before it reaches the users.
01:57And four, neuro-symbolic guardians, you rule life in Python, not in the prompt, and the model cannot escape them.
02:08So, five, runtime guardians, because if you don't want to block, you can steer, and you don't you don't need to block everything.
02:18And when a rule fires, the agent self-correct and complete the task.
02:26No hard stop, no user advice.
02:29So, for each technique, I will show you the agent without it, and then with it, so you can compare.
02:38And all the demos uh I'm using a travel agent that I built using a Strands agent.
02:45And a Strands agent is a open-source agent framework that we maintain on AWS.
02:56And I am Elizabeth Fuentes Leon.
02:59I am a developer advocate for AWS.
03:02I'm focused on agentic application.
03:05And here, in this QR code, you will find everything that you will need to recreate all these techniques that I'm going to show you in a moment.
03:21So, let's get into Let's get into it.
03:31So, semantic tool selection, our travel agent has 29 tools.
03:39Flights, hotels, payments, weather, cancellations, all all our dummy tools are not like a travel agent for real, but every time that user sends a message, all the 29 tools description
03:56go into the context windows.
04:00The model reads all of them before deciding what to do.
04:07And if your agent has memory, the rows to every conversation adds more context that gets sent with every single message.
04:20And you pay for every single one of those tokens, whether the model ends up using the tool or not.
04:30To understand where those tokens come from, you need to see what a tool actually looks like to the model.
04:39In a Strands, you write a function with the tool decorator that's the tool, and a name, a description, and a docstring typed parameters.
04:52Then, Strands take that and generate a schema with name, description, parameters, and the schema is what goes into the context windows on every call.
05:07Each tool schema is about 17 or 200 tokens, depending of how many parameters it has.
05:17If our travel agent
05:18[snorts]
05:19has 29 tools, that adds up to somewhere around 3,000 tokens per call, just for the tool description.
05:31Before your message, before the response, every single call.
05:38By creating a tool database, we can filter the tools that the agent may need before the agent the symbol.
05:48With this filter, the model sees only three most relevant tools.
05:55Tokens usage drops from thousands to fewer than 300.
06:02Let me show you in the code.
06:05Here in my cure ID, this is the Let me clear all the outputs.
06:12So, first we need to install the requirements.
06:16Here is the requirement file.
06:18This is a Jupyter notebook because it's more simple to show everything, but there is one application that you can run if it is more comfortable for you.
06:27So, here are the requirements.
06:29I have the A strands agent and because I'm using open AI as a model invocation, I'm using the API from open AI.
06:38And I need a strands agent for open AI.
06:41You can use open AI strands agent with all our my god, almost all the model provider.
06:47And with a strand we Amazon better of course because we as AWS, we are the maintaining of this framework.
06:55And to use a Amazon better, you don't need to add the model provider.
07:02And because I need to embed in I need to create embeddings for my vector tools database, I'm using the sentence transformer.
07:12This is a super simple model that runs locally, so it's free.
07:17So, you want you can run this locally you computer without expense another model embedding.
07:24And you can use a strands with all llama too.
07:28So, if you have a local model, you can run everything in this in your computer for free without expense any tokens.
07:38And I use it files as my vector store, local, super simple.
07:45And well, right now I'm going to use a Neo4j.
07:47This is for other demo that I'm going to show you in a moment.
07:52And because I'm using some environments, I'm using Python of And let's see.
07:57So here I want to run this.
07:59Let me put it bigger.
08:01I know that you have it some problems.
08:03Bigger.
08:04Little more little less.
08:06And here, this is better.
08:10So.
08:11I style my requirement.
08:12I already did that.
08:14And I'm using OpenAI, so I need my my API.
08:18My API key.
08:19API key.
08:21And here, I invoke my strands.
08:23I need the agent because I want to be like agent that I'm using OpenAI.
08:27And I have a bunch of tool dummy tools here.
08:31I let I going to show you in a moment.
08:33And I want to use this because I have some um functions that I'm using to uh create the build the index for my vector store.
08:44And I need the search tools when I'm using the vector store, I put my query there.
08:49I search for the I search for the tool that I'm going to use using vector um um search for vectors.
08:57And I swap the tools.
08:59I going to show you that in a moment.
09:01So let me show you all my dummies tools.
09:03Where is the engage register here.
09:08Here I have all my tools.
09:10Uh this is for swap.
09:12Search, build index.
09:14And where are my tools?
09:16Shoo shoo shoo shoo shoo here.
09:18So here are all the dummy tools I created for this.
09:21You know, it's super.
09:22This is a demo, please.
09:24It's not something that you can use to go to production.
09:28No, please.
09:29And okay.
09:30So where is this this one?
09:32So I run this.
09:34And I build my semantic index.
09:36I already did that via Yeah.
09:39So, I have 29 tools here, 29 dummy tools, and I want to test this with a lot of different queries, and this is the ground truth.
09:49I have a ground truth because I know what is the best um the best tool to answer the question, so we can know if our agent is okay or not okay.
10:00So, let's run this.
10:02We have 90 queries and 29 tools.
10:06So, this is some helper functions.
10:09I don't care.
10:10Yeah, I care, but I don't want to explain you that.
10:13So, this is my agent, my traditional agent, where I'm putting all my 29 tools inside the agent.
10:20I have my model, some helper function.
10:24This is my agent.
10:25I only This is the way I create a agent using a strange agent.
10:29I put all my tools, and my system prompt is you are a travel assistant.
10:34You select correct tool to answer question.
10:36Oh my god, super.
10:38And the model.
10:40Let's run this.
10:41And this is whatever.
10:43This is some function to count the tokens because in inside this strange agent, you can you can count the tokens.
10:51You You can know how many tokens are using for in and out the the agent.
10:57This is some helper function, and yeah.
11:00This is Every time that I put a uh a prompt inside this agent, I'm only sending the prompt, and the agent respond.
11:09Then, I I use a new agent because it's a a fork.
11:13So, I'm not having a conversation with the agent.
11:15I only send a uh a question, and I receiving an answer.
11:20So, for each question, it is spend around 2,000 tokens.
11:25So, in and not always it give me the right answer.
11:30Here.
11:31So, yes, I have um I'm not so good at accuracy, and the average is 1 1,000 tokens.
11:40Now, let's use my new AI with the semantic approach.
11:44The thing The first thing that I do is for every single query, for every single prompt, I send the query to my search tools, and I going to receive
11:56the top two the top K three most relevant with more probability to response my query because this is a semantic search inside my vector store.
12:08And then I use that response.
12:12I going to select that name, and I going to put that tools inside my AI.
12:18So, I only going to use the three tools that this semantic search retrieve me for my AI, and I send that only tools.
12:28And this is my function helper, and that's it.
12:31So, I going to send the all the queries again the same.
12:35And for the first question, I have 4,000 tokens three blah blah blah.
12:41And yeah, we have a huge different because I don't send the all the 29 to tools in all the queries.
12:51So, yeah, bam bam.
12:53And okay, semantic memory.
12:57Yeah, okay, now come on.
12:59Finish.
13:00So, let's let's let's go to the next one.
13:03So, this engine is only sending a question, and I receive a response.
13:08I don't have any conversation with this engine.
13:11What happen if I start to have a conversations?
13:14If the engine remember me?
13:16So, we need to we need to send only the tools that this engine is going to use.
13:22Because if I give the all the tools I'm using in the conversation history, uh then I it's going to be in a moment that I want to have the 29 tools inside my engine.
13:32So, we are not resolving the we are not having a a um We are not resolving our problems in the engine when we having a conversation with the engine.
13:44So, we need to put the tools and then we need to remove the tools.
13:49And we can do that with the swap tools.
13:53And let's run this.
13:55And what swap tools I got run this, no. Let me Let me show you first the swap tools functions here the register.
14:06I delete this because with a strands in each invocation, you have complete control of the status because this is a a gentle loop and you can we use this a gentle loop
14:23how for because this is a gentle loop and you can do almost whatever you want.
14:33You can put and remove everything inside the loop with some few lines of code.
14:40So, let's see.
14:41Here we have the engine.
14:43We have the tool register through your through tool registers.
14:47It's something inside the engine status.
14:51So, we can clear the tools and that's it.
14:55So, in the next invocation, we can clear the all tools and we can add the new tools.
15:03Here, where is my I have a lot of things here.
15:09So, I run this.
15:16It's going to take a moment.
15:28And yes, we can see that in each invocation, my amount of talking is increasing.
15:35Why I have more?
15:37Because I'm sending the chat history as well.
15:40So, I send the tools that I need, only the tools that I need, and the chat history.
15:46So, that's why I can see that my amount of token is bigger and bigger.
15:59And the see, the accuracy is better.
16:03And yeah, we have more tokens.
16:08So, yes, probably we don't have the best accuracy here because this is a super demo with super dummies tools, and some queries are ambiguous on purpose because search for something,
16:23for something, check something.
16:25And the demo has a generic tools, dummy tools, with some similar name.
16:32And when all the 29 tools are visible, the model sometimes pick the wrong one.
16:40And with the filtering, those generic tools uh only appears in the query actually match them.
16:48So, you can run everything and you can test this.
16:52So, this is only that run only locally.
16:55But, what happen when you want to put this demo in production?
17:00You can of course, you can build a bigger, better store.
17:05I don't know, you can use Postgres.
17:07I think it's too much.
17:08But, we on AWS, we have Amazon Bedrock Agent Core.
17:14So, Amazon Bedrock Agent Core is a service dedicated only to agents in production.
17:22And in size of Agent Core, we have Agent Core Gateway.
17:27So, Agent Core Gateway Gateway allows you to build this index [clears throat] uh alone.
17:36So, you only have to say, "Hey, this is my tools."
17:40And Agent Core Gateway is going to build everything for you.
17:43And it can have the vector search inside.
17:48So, the routing layer is inside the Agent Core and it handles the tool selection automatically.
17:54So, you register your tools once and it find the right one for each request.
18:01It's the same principle but without infrastructure to manage.
18:06Now, let's see the next one.
18:10GraphRAG.
18:12You know, RAG is retrieval augmented generation.
18:16Is how agents access your own data.
18:20You take the user question, search your documents for the most similar content using vector search and pass what you find to the model.
18:31The model answer from that, it's work well for open question.
18:37Find me something about this topic.
18:39But there is a category of question where that breaks down.
18:44What is the average rating across all hotels in Paris?
18:48How many hotels have a pool?
18:51Vector search always returns something even when nothing is truly relevant.
18:59And the agent only sees the top end chunks of your all data at a time.
19:06It cannot aggregate, count, or traverse relationship across all the full data set.
19:12So, it estimates.
19:15And it present that estimates as a real fact, you know, a real answer.
19:22So, here we have graph you build a vector store and it retrieves three chunks from 300 documents and the model guesses.
19:34If we are using graph rack, you can run a query across all the and returns a compute results.
19:43Graph address this differently because instead of a retrieving text chunk, you can build a knowledge graph from the documents.
19:55Knows relationship and structured data.
20:00For the demo, I'm going to use Neo4j locally and the model is going to write a Cypher query to search it.
20:10Cypher query is Neo4j query language.
20:14It's similar to SQL.
20:16So, the graph run that query across all the data.
20:20And the model gets back a compute verified results, not a sample.
20:27And before running this demo, I'm going to start the dependents.
20:32Let me show you.
20:33Let me Let's go to the code.
20:37So, graph This is the notebook I'm going to share with you.
20:42So, we need to install the requirements again.
20:45What we have here, let me see.
20:48This is the first one, yeah.
20:50Requirements.
20:51I have I'm going to use OpenAI again.
20:55Let me just close this.
20:56I'm going to use Neo4j and we need Neo4j for graph rack.
21:01And for the engine that is using normal rack, we are going to be a super simple uh vector store in files and we are going to use again the sentence transformer.
21:15And let's see.
21:16I already have all right, I invoke [clears throat] my open AI.
21:21This is my tools.
21:22I I because I want to create some tools here, I want to use open AI and I use in graph database.
21:30And this is my I have my Neo4j locally.
21:34And let's run this.
21:37And this is something to check Neo4j.
21:39So, I'm building my uh vector store.
21:43This is my files and this is my Neo4j that I have here locally.
21:47I don't know if I can show you.
21:48I think not.
21:49And I have a tool, a normal tool created with a decorator.
21:54This is to search inside the my vector store.
21:58You see that's super I I have the query.
22:00I I create I I create the the embedding for my query and then I search inside the vector store and I have the query for the knowledge graph.
22:10So, here the model need to understand that to search in the query in the graph knowledge base, it need to build a Cypher query.
22:20So, I put that in the context because we already know how to build uh tools, right?
22:25I put that in the context.
22:27I have my driver to send the data to uh to connect with the vector store and to send the Cypher query that the model is going to create for me.
22:37And this is to read the results and that's it.
22:41That's the only tool that I need to search in my knowledge graph.
22:44Okay, this is my model open AI.
22:48This is my rag.
22:50My rag engine and uh this is my graph engine.
22:52I have two different engine to compare the results.
22:56Let's see.
22:57Well, I have a problem.
22:59Yeah, I'm ready.
23:00So, let's do this round the first test.
23:04Aggregation.
23:10Here, what is the question?
23:13What is the average guest rating across all hotels in Paris?
23:17So, meanwhile this engine is Oh, it's already done.
23:21So, what I have here I have the average guest rating across the hotels listed in Paris is Um it calculate, you know.
23:31When something When I'm using rag, it go to the to the vector store, it receive the n uh possible answers for this question.
23:41Because this is an aggregation, it's going to use the data that it receive to build a mathematical uh operation.
23:50This is the thing that I have here.
23:52So, everything that we see here is the is the model reasoning.
23:56So, when I run the graph engine on the mean, the engine is only going to give me the answer, you know, only these tokens here.
24:06Because the Cypher query, it can give me the mathematical operation itself.
24:12It can do that.
24:13So, we don't need the engines, the LLMs, the model uh do that for me because the Cypher query already give me the right question.
24:22So, it is 47 and the average.
24:26So, here in this uh first we are lucky because probably they are Yeah, only two hotels.
24:35But, what happened if this vector store have more than two or three hotels in the in the vector store?
24:44So, we are not going to give a real answer.
24:47It's going to the LLM is going to calculate with the only three answer that it receive.
24:54So, it's going to build this operation with only three hotels.
24:58But, if the vector store have more than three, we are going to have some problems.
25:05Something that I am you know uh yeah.
25:09So, let's run the next one.
25:11The prices counting.
25:13Something similar too.
25:14How many hotel have a swimming pools as a amenity?
25:19How many again?
25:21So, it give me the the traditional It's appear that the search did not return some specific information about hotels in Paris.
25:30Okay.
25:31What happened with the other one?
25:33There are currently no hotel that offer swimming pools.
25:37No I. There is no hotels.
25:40So, the other one is like mm would you like to ask about hotel with other specific amenity or information?
25:46It's like mm maybe it is or maybe I don't know.
25:50Yeah, it's not so accurate, okay?
25:54So, mostly conversation.
25:58Oh, what is my question?
26:00What are the room types and price for the highest rating hotels?
26:05What are the rooms types and price?
26:08This two question.
26:10So, it search the fact I had had only one.
26:14The highly rated hotels in any company Paris blah blah blah.
26:17I I don't speak French.
26:19With the guest rating is that one.
26:22However, I currently don't have Okay, blah blah blah.
26:26A lot of data there.
26:27So, what happened with the other one?
26:30Receiving notification for Oh, I have a error.
26:35I hear in the second one I receive the the question.
26:38The highly rated hotel is harmony blah blah blah.
26:40They offer following types.
26:42But, unfortunately these rooms are not available.
26:45So, I receiving a answer.
26:48I don't receive a lot of blah blah blah there.
26:50I only receive the answer what I need.
26:54And for the next one, how was the meditation?
26:59What is the question?
27:00Tell me about hotels in Antarctica.
27:03A spoiler, there is no hotel in Antarctica.
27:06There is zero hotels in Antarctica and let's see.
27:10It's appear that the certain order to specific information about hotel Okay, because they are I search for the other have the tail blah blah blah in the If you are looking for particular that experience
27:22specifically inquire about visiting Antarctica.
27:25No. Please let me know and I can assist you for it.
27:29Okay, a lot of tokens that I spending there in that answer for Elena.
27:34What happened with the other one?
27:36They are currently no hotel listed in Antarctica.
27:39Of course, because it create a cyber query, the cyber query and it receive zero.
27:45So, it no, it give me a honest answer.
27:49Here's summary that close grade for me for this Neo4j notebook.
27:55But here is something that I want to show you before go to the other one.
28:00So, here something that I love for Neo4j because why I'm using Neo4j?
28:07Because in the library the Neo4j give me uh it can build it use a LLM.
28:16I'm using the OpenAI as well to build the knowledge graph.
28:21So, how I build my knowledge graph?
28:24I only have a bunch of data, a bunch of txt only text and I send that data to the Neo4j library that I miss out.
28:36Okay, I send that data to this and Neo4j using this all this um library just right here, it can understand all my data and it can build the graph for me.
28:51So, I don't need to create that using the simple knowledge graph pipeline inside the knowledge graph library.
29:00So, that's why I'm using Neo4j.
29:03It's amazing.
29:04It's super simple to use.
29:06So, I invite you to use it at that.
29:08So, let's go to the next one.
29:13So, multi-agent validation.
29:17Sometimes an agent fails, and nobody find out.
29:21It calls a tool, and the tool returns an error, and the agent does not surface that error.
29:28It generate a confidence success response instead.
29:32The user thinks it work.
29:34You think it work.
29:36It didn't not.
29:37The agent acts and validate its own output in the same loop.
29:43There's no separation, no second opinion.
29:47So, when something goes wrong, it rationalizes and tells you it's okay.
29:53It work.
29:55Here is what happens.
29:57Inside a single agent, when it fail, it calls the tool, gets an error, rationalizes it, and returns a success response.
30:10The user never sees the error.
30:13You can address this by adding a validation layer.
30:19You can have three agents in sequence.
30:24One acts, one checks, and other approve or rejects.
30:30Strands agent has a built-in class for this called swarm.
30:36It manage the handoff between agents automatically.
30:42You just the role of each agent in a system prompt.
30:48And let me show you that.
30:53So, this demo only needs a Swarms agent with open AI integration.
30:58Let's see the requirements here.
31:01All you need of is Swarms agent.
31:03We don't need anything more.
31:04Let's go to the notebook here.
31:08And the key important thing here is the already running is Let me see here.
31:16Is the Swarm.
31:18The Swarm it's what lets you to connect multiple agents together without creating like a for a while manually to put the all these agents together.
31:30It can build a chain and manage the handoff between them automatically.
31:37So, here we are going to create three agents.
31:40Let me go down because we have the normal and we have some ground truth data.
31:47So, here is to create a a single agent.
31:49So, we have we are going to test three different scenarios to validate a booking.
31:54This They we expect true to enable hotels.
31:58We expect false and non-existing hotels and missing booking.
32:02We expect false, too.
32:03This is how we created single agent.
32:06We need a prompt, some tools.
32:08And we are using the tools that we have here.
32:12And here we know that this is the data.
32:15So, the agents can can give out the answer that we are looking for.
32:20And that's it.
32:21Let me go to the other ones.
32:23Oh, wait here.
32:25So, how we build the Swarm?
32:27I already run this.
32:29I want to show you this one.
32:30So, we are going to build three different agents.
32:33The executor, the validator and the critic.
32:37The we have a system prompt.
32:40You are executor agent for a hotel booking system.
32:43[clears throat]
32:44Use the provide tools to fulfill requests accurately and blah blah blah.
32:49The validator is you are a validator agent.
32:52Review what the executor did and output exactly on off.
32:58And we have the critics that is going to set here approve or no reject.
33:03Let's run this.
33:05And oh, what is here?
33:06Our model didn't define the model.
33:08Sorry.
33:09Sorry, please.
33:10Forgive me the life because I didn't run this one.
33:17Let's run now because this is the the normal.
33:21And the model is there, right?
33:24What?
33:25A little bias.
33:26Run that.
33:29What's happening with you?
33:30Let me go this.
33:31Let me copy and paste.
33:33This one.
33:34I don't know why this give me problem.
33:39Multi-agent swarm is the time the same name.
33:41Come on.
33:43Yeah, you can see this is life.
33:44I don't going to edit this.
33:47So, yes.
33:49Okay, we have Thank you.
33:53And the swarm, how we how we create the swarm?
33:56We have swarm and the swarm function, we put all the agents together and the entry point is the executor.
34:05So, all everything is going to start in the executor and then it's going to hand off.
34:10And six time because you can handle that, too.
34:13So, get the swarm final response.
34:15Let's do a for and they send all the response.
34:18And book the Grand Hotel.
34:22I have booked a Grand Hotel for Alice tonight.
34:24Booking blah blah blah.
34:25Hand off.
34:27Valid.
34:28Okay.
34:29Looks good.
34:30Critic, approve.
34:32So, the critics is approving the this.
34:36This is the other one.
34:38And we can run some comparison here if you want.
34:43So, here we can see that the swarm handles the flow between the between all the uh three different saying.
34:53It was what happens when the single agent tries to conflict something that does not exist in the system.
35:00And now the same request through the swarm, the executor gets the error, the validator catches, the critic rejects this, and the user never see a fabricator response.
35:13So, you can we can test here with the single agent that it's a suggested unknown entry and return success.
35:23And the swarm, it have a executor that go the error, validate saying, "Hey, come on, man.
35:30Hallucination and the critic they reject."
35:32So, the user sees a clear failure.
35:36The fabricator confirmation so missing the issue here.
35:41And you can run this to compare the two different agents.
35:46So, here we can see that the single agent it have everything like a yeah, it's okay, it's correct.
35:53But in the multi-agent swarm, you can see that the last one is a you know, missing issue here.
36:00Something is happening.
36:02Now, let's go to the next one.
36:05Okay.
36:06Neuro-symbolic Guardians.
36:09You have a rule for your agent.
36:11Let's uh say maximum 10 guesses per reservation.
36:15You write in in the system prompt.
36:18You even write in in the tool description.
36:22And the agent still calls the tool with 15.
36:26No, because it is ignoring you.
36:29Because prompts probably are suggestions, not constraints.
36:34The model process them as a text.
36:38Not as a logic it has to execute.
36:42It's probabilistic.
36:44Only code execute logic.
36:47A rule in the prompt the model read it as a suggestion.
36:52A rule in the code the model can not escape it.
36:56Neuro-symbolic guardians rules put the rules in the code.
37:01E-strands A and has a few that call him hooks.
37:06The the function that is trans called automatically at specific moments in the A and loop.
37:14It is this case right before a tool executes.
37:18So you have a hook.
37:20You write the rule, check the parameters and if they fails you cancel the call.
37:27Let me show you this in the code.
37:31Here we have the Jupiter notebook.
37:34We have the requirements here.
37:37Um what is the same requirements as the previous demos we only need I will install everything again.
37:42We only need E-strands and open AI.
37:45We don't need anything else.
37:48And um my API.
37:53And here is the thing important.
37:56In this demo are these three hooks provided in this trans base class.
38:02So you allows you to create the hook.
38:04So we have hook provider, hook register and before to call event.
38:10And the tool register is what E-strands pass you to register your call back.
38:17And before to call event is the event that fires every time the model is about to execute shadow.
38:27That last one is what makes it possible the intercept the call before it runs.
38:34So, you have the before two calls even and call we have after two calls even that we are not going to use that here.
38:42And let's see the rule in the prompt.
38:45Remember, the model reads as a text and it may follow or not follow them.
38:52So, let's see this.
38:53I want to run this.
38:55So, here we have some similar state that we are going to use in this engine.
39:00And let me this is the symbolic rule.
39:05So, booking rule if if rule booking rule as a this is something that we create.
39:12Let me show you the rules here.
39:14So, we have the rules.
39:17Rule one.
39:18Validate dates.
39:20Check in must be before check out.
39:22If what is validated they check out.
39:25This is something that it check.
39:27This is a something that is going to invoke when the rule is is uh is must have to use it.
39:36And uh to do We have another rule for max guesses.
39:45Maximum take guesses per booking.
39:47So, if I want to use I don't know I if I want to book in uh uh I want to do a booking for 11, it's going to block me.
39:56It's not going to reject the booking.
40:00And we have some confirmation rule.
40:02Payment before confirm.
40:04You know, you can confirm if you don't have the payment.
40:07Cancellation rule.
40:08Cancellation within the cannot cancel within 48 hours of check in.
40:14So, this is something that I've created here, you know, to this demo.
40:18So, I have booking rules and I have confirmation rules.
40:21I don't I use in here the cancellation rule.
40:24So, create validation hook.
40:27We create the validation hooks here, right?
40:31Like this, neuro-symbolic rule, and we use the hook provider.
40:35So, we have a bunch of code here.
40:38Then, we add the booking rules, the confirmation rules, and the state.
40:45And you can check this by yourself later, but we want to see the demo running.
40:50So, that we define the claim tools that we are going to use them for the booking hotels and the process payment.
40:58This is the the normal tools, you know, and we need to add that tools to the hooks.
41:03I think I add that.
41:07Yeah.
41:08Tool name, book hotel.
41:10So, this thing is going to get the eight when the book hotel is using the Yeah, the book hotel.
41:21Okay?
41:22So, let's create the agents for the comparison.
41:25So, we are going as the other demos, we have three different scenarios.
41:31Confirm booking without payment.
41:33Payment must be verified before the confirmation.
41:36This is the rule that I must trigger this.
41:38We have booking hotel exceeding guest limit, and we have valid booking for five guests.
41:46So, yeah, we have the three scenarios.
41:49And we have the normal engine, the normal engine, and the vaseline engine.
41:54And we have the engine with the neuro the neuro-symbolic guardians.
41:58So, we have the hook, the neuro-symbolic hook.
42:02Uh this is the same.
42:04Here, yeah.
42:05And we add this If you can see, the normal engine
42:11[snorts]
42:11is only three lines.
42:12The tools, the model, and this is the line that give us the difference between these two engines.
42:19So, I already run that.
42:21Now, let's go to confirm a booking without payment.
42:29So, I run this my engines.
42:31The first one is Abis.
42:33Come on, man.
42:34I Yeah, because I Yeah, it confirmed the booking without payment because it doesn't have any rules and the promise is super basic.
42:41And the other one is Let's open this and is booking blocked.
42:49Payment must be verified before the confirmation.
42:52Thank you.
42:53So, it's okay.
42:55Now, let's run the other one.
42:57Test second scenario.
43:01What is the question?
43:03Booking hotel access guest limit.
43:06So, the question is I Yeah, I have more than So, here the hotel Abis successful booking for Yeah, because the promise is basic and I don't put in any rule there.
43:20And for the other one, it seemed that the Grand Hotel has maximum capacity of 10 guests per booking.
43:25Additional booking must be made at last 1 day in advance.
43:30I don't remember the day.
43:31Probably This is something that they give them.
43:34And to validate booking is both engines execute all rules pass.
43:39Because they Yeah, it's only a validation for booking.
43:42So, book hotel for blah blah blah.
43:45All rules passes.
43:47It seemed that the booking is still needs to make.
43:49I don't I didn't remember the question.
43:52So, booking is made provide I Yeah.
43:55Okay.
43:57Okay, I get it.
43:58And Now, run all the thing this question.
44:04Confirming without payment, booking save max.
44:06And what happened here?
44:07This is all the scenarios that we just run.
44:10So, this is awesome thing to compare the results.
44:12So, Bali booking king secure five guesses, allowance, correct, wrong.
44:1950, blocked, wrong, correct.
44:23Confirm, so there we have here the comparison between the two ends.
44:33So, what we have what we have here is same model, same tools, same prompt.
44:41And the different that outcome because the rules are in Python knowing the prompt.
44:47This pattern of enforcing rules in code before the tools run is also what Amazon Aion core policies service that we have does at the structural level.
45:01So, the same concept but managed for you in production.
45:06And you only have to create the rules.
45:08So, the hooks are all or nothing.
45:12They block everything or approve.
45:16But sometimes you want the agent to adjust and keep going.
45:22No and stop and leave the user waiting.
45:26That is what I will show you in the next.
45:32Run time guardians, run time is still is still don't block.
45:38This is the next one.
45:40Hooks blocks unconditionally.
45:44The agent is stop and the user has to retry.
45:48For a hard constraint, that is exactly what you want.
45:53But sometimes the rule is soft.
45:57Maybe room fits for four guests, but a group of six could go book two different rooms.
46:08Or a flight is full, but there is ability on the next one.
46:13You do No, you don't want to block everything.
46:17Probably, you want the agent to find a option and complete the task.
46:23That is a steering.
46:25Here, we have the hook that fires and the task failed.
46:31And with agent control, it steered the models and the task complete.
46:37The other different here is operational.
46:40Because with the hooks, it's uh changing a rule means changing code and redeploy the whole harness, whole the engine.
46:49And with the agent control, which is the name of the open source library that we are going to use here, rules are registered on a local server via API.
47:02You open the agent without touching the agent code because the agent picks them up immediately.
47:10Let me show you that here in the code.
47:14Yeah, this is the notebook.
47:17In this demo, we only need a extra packet.
47:21The agent control is the key.
47:26So, the agent control is the one that helps us to create a steering.
47:32And here, we are using the setup control.
47:36This is a a little application, a little app that I create with the local server and the steering rules.
47:46Here, we have the local server.
47:48We have the control.
47:50They
47:50[snorts]
47:50are the steering rules.
47:51First, we have the the steer master guest.
47:56A steer, you know, guy.
47:58So, guy agent to reduce a guest count when exceeding maximum of 10.
48:06And he's going to steer.
48:08And we have some control that deny.
48:12For example, deny no payment.
48:14Block blocking confirmation without PR payment.
48:20And here, well, this is to to create the um the service.
48:26The service.
48:27And let's go to the uh notebook.
48:32So, I'm ready.
48:33Um Let me go here.
48:36Yeah.
48:36So, I need the environment.
48:40So, this is the agent.
48:43Um I'm going to have a error here.
48:46Wait.
48:46Wait.
48:46Wait.
48:47I'm going to use bedroom.
48:48Wait.
48:49Wait.
48:50No, then let me comment this one because this going to give me a error.
48:57And let's run this.
49:00Yeah, so we have the hooks as we did the demo.
49:04So, we have book any company Lisbon and we have some prompts.
49:08You are a hotel booking assistant.
49:10When booking, first describe what you will book in blah blah.
49:14And this is the prompt for my agent.
49:17Where is my agent?
49:18Agent control.
49:19This is for the Well, this is on some helper.
49:23And this is my hook that you already know because we create this.
49:27So, we have the system prompt.
49:29And we have the hooks.
49:31So, let's test this agent with the uh book any company Lisbon for 60 guests.
49:38And if you remember, this only can book for uh less than 10 guesses.
49:46And of course, he's blocking.
49:49Now, let's go to the AIN control new agent.
49:54Here we have two key importances or two key imports that are important for the AIN control SDK.
50:02So, we have the AIN control plugin that captures AIN events and sends them to the AIN control server.
50:11And we have the AIN control student handle that listen for a student decision for the server and delivers them back to the model.
50:21Together here they are what connects a student to the AIN control student logic.
50:29So, let's go this with this.
50:35So, the student agent is going to a river room for any company list board 50 guests.
50:45Let's see what it's doing.
50:47Yeah, I have to set for booking for a stay in the company list board for 50 guests for May.
50:53So, it's the reservation have been split into two rooms.
50:57So, it took that it in self.
51:00You know?
51:01One room for one and other room for five and that's it.
51:07So, use hook for hard constraints.
51:11AIN control for software
51:13[snorts]
51:13rule.
51:15Now, we have five technique or running locally.
51:20But, how do you take this to production without maintaining service, without building infrastructure?
51:29Let me show you how.
51:33Everything that I just built here in the previous demos runs locally.
51:39For the production version, Amazon Bedrock Bedrock Amazon Bedrock AIN core give you the run time.
51:47A gateway, a short term and a long term memory, a cloud watch observability, build in no servers to manage.
51:58Here the architecture, the strands engine runs inside the run time.
52:03And you know, you inside the run time, you can put every framework that you want, not only run a strands.
52:10The gateway rules tools calls to the lambda function automatically as a tools.
52:16And the steering rules from the previous demo live in the DynamoDB.
52:22So, you change them there and they are live on the next call.
52:29And you don't need to re-deploy anything.
52:33And you want to use Neo4j, of course, you can use Neo4j Aura DB, that is a external graph database, which also has a free tier.
52:44And the code is in the repo here.
52:51Is in the repo.
52:53And you will need AWS credentials if you're using Amazon Bella Betray in core.
53:00And in the resource link, there is some credits.
53:04I hope that you can find it because I always try to to give away some credit for AWS so you can deploy everything for free.
53:15And uh if you are new here in AWS, I have a repository that you can use to deploy everything over this architecture uh using a notebook, too.
53:27But, if you are familiar with CDK, cloud developer kit, you can do it that as well to deploy everything at once.
53:35Uh both options are in the repository, of course.
53:38And you can go deeper in Agent Core with all the documentation that I left there.
53:45So, please don't stop at the demo and try going to production with Amazon better agent car.
53:52So, let me bring it all back.
53:56Tokens waste on every request.
53:59You can fix it with semantic tool selection.
54:03And confident answer it's never computed.
54:07Possibly it's possible because you are asking how many.
54:12So, you can use graph rack and query the data.
54:16Don't sample it.
54:18Sometimes we have fabric case success confirmation.
54:21You can use multi-agent validation and second pass catch it.
54:27And rules the models uh quite a little skip it.
54:32You can use uh neurosymbolic guardians to enforce in the code.
54:39Don't trust in the prompt.
54:40And for the hard blocks that I stopped the user, you can use runtime SDR, self-correct, and finish.
54:51Each demo that I just showed you is in the in the repository as a notebook and an application as well.
54:58Start with the demo one and then go to the demo five.
55:03And if you want, deploy it in the productions.
55:07So, have you tried any of these in your own hands?
55:12Thank you to join me in this session and happy building.