Stop Giving Your AI Agents API Keys - Kim Maida, Keycard & Mason Egger, Temporal

AI Engineer · 52 min · 484 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:12My name is Kim Maida.
00:14I work at Keycard and Keycard provides agent access identity and access at runtime for agents.
00:24Oh my.
00:25Hi everyone.
00:26My name is Mason Egger.
00:27I'm a solutions architect at Temporal and Temporal provides durable execution to your software applications up into and including agents.
00:35So, not just agents, but pretty much the entire the entire stack.
00:38So.
00:43So, everything you're going to need for the workshop is at the link that's on the screen right now.
00:47You actually do not need to sign up for Keycard ahead of time because the sign up will be included like within the curriculum, but in order to get an account for your own personal use, if you want to play with Keycard on your own after the workshop,
01:02you can go to the link that is provided at the bottom and enter that registration code in order to sign up for a personal account.
01:16Okay.
01:17So, we'll come back to this slide, but I am going to start out by talking a little bit about agent access.
01:26So, one of the things that we do a lot of these days is give API keys to agents to give them access to resources, right?
01:34And there's several problems with doing things this way.
01:36One being credentials aren't attributed to a user identity or an agent identity.
01:42The agent has unrestricted access to any and all permissions in that API key.
01:48And finally, we can't just sort of stick human in the loop everywhere because humans make mistakes, too, and many agents also run autonomously.
01:59So, the way we solve this Keycard is by using an open standard called token exchange.
02:04This This standard has actually been around since 2020 and it extends OAuth 2 and I am going to show you what we're going to be doing today.
02:16So, in order to give identity to an agent and the user behind the agent, we need to know who the user is as first order of business.
02:24So, an authorization server verifies identities and issues tokens.
02:28Identity providers like Google, Okta, Auth0, etc. provide authorization servers.
02:33And if we want to get an identity chain in the agent execution path, then we need the user to log in first.
02:42After logging in, the authorization server prompts user for their consent to delegate some portion of their access to resources with a subset of their permissions called scopes.
02:53Now, this is kind of the first narrowing of access, right?
02:55We're only delegating some of the user's total access to the agent.
03:01And the authorization server issues a token I that identifies the user and includes their level of access.
03:10So, now if we go into what actually happens in the agent execution path, we have an authenticated user now with a known identity who wants to use an LLM to interact with resources.
03:22Now, an agent is a control loop that calls an LLM and in this workshop, that agent is an expense desk agent.
03:29That's what we're going to be working with.
03:32It's going to call in an expense MCP server which sits between the agent and a ledger API.
03:38This provides the agent with a defined set of tools and it also provides the connection to that downstream resource.
03:47Now, an MCP client takes the agent's proposed tool calls and it dispatches them to its MCP server.
03:55The runtime is a process that runs with the agent loop and executes calls.
03:59So, in our case today, the Expense Desk application is the runtime.
04:06We're going to use a LangChain agent that's implemented in this custom Expense Desk app.
04:13So, if we follow a prompt through this execution path, the user submits the prompt to the runtime, which calls the model and sends it to the LLM.
04:21The model proposes tool calls, which are then dispatched by the MCP client to its MCP server, which executes the tools, and then calls the resource API.
04:32The API returns a response to the MCP server.
04:36The MCP client delivers the response to the runtime, which calls the model, and this loop repeats until the model is satisfied and returns the response to the user.
04:50Now, if you remember, we have an authenticated user with a token identifying who they are.
04:55And this token becomes what we call a subject token as identifies the subject on whose behalf the agent is going to act.
05:03So, in order to support token exchange, we need an OAuth client that's capable of executing code.
05:09So, that might be a gateway between the MCP client and maybe a third-party MCP server, or a custom agent app, or a CLI wrapper around an off-the-shelf coding agent.
05:19In this case today, the Expense Desk agent is a custom agent app that implements token exchange for access to the Expense MCP server.
05:28This MCP server also supports token exchange for the next downstream resource, which is the ledger API.
05:35This connects to the database where the expenses are actually going to be stored.
05:41Now, the prompt and the subject token are sent to the Expense Desk agent app.
05:45The agent loop runs, and the model proposes a tool call.
05:52The Expense Desk agent authenticates with Keycloak's security token service.
05:56Now, this is a special authorization server.
05:59And the Expense Desk agent can authenticate using its client credentials or a workload identity.
06:07It also sends the user's subject token that declares the user's identity and their level of authorization.
06:14Now the expense desk OAuth client creates a token exchange request asking permission to access the MCP server for that tool call, but only that one specific tool call.
06:25So now we basically have three pieces of information.
06:29We have the identity of the agent that's requesting access.
06:33We have the identity of the user on whose behalf the agent is acting.
06:37And we have the level of access that the user granted to the agent.
06:43So now we need to decide if the requested token should be issued or not.
06:47And we can do this with policy, which is evaluated against what access is requested from what resource, who the agent is, and who the agent is acting for.
07:00Now if the delegation chain and the requested access all fall within the policy, then Keycard issues an access token that can be only only be used at the MCP server.
07:11Now this issued token usually expires within minutes and you should never store it.
07:18The token is sent to the expense desk MCP client, which makes a tool call to the expense MCP server using that token as a bearer credential.
07:26The MCP server validates that token and then uses it to authorize a tool call.
07:33The tools need access to the ledger API resource.
07:37So at that point the MCP server then initiates a second hop.
07:41This is where it sends its own token exchange request to Keycard.
07:46And the request to exchange that token is received from the expense desk for a token to access the ledger API, which is the downstream resource.
07:59Now Keycard evaluates request against policy and then mints the credential for the resource scoped only for the access that's needed for that specific tool call.
08:11The MCP server receives this token and then it uses it to call the ledger API.
08:18The API validates the token and then grants the appropriate requested access.
08:23Now, the MCP server never stores the token and it discards it as soon as the call is done.
08:29The API response then flows back up and the agentic loop repeats until the model returns the answer to the user.
08:39So, at a high level, this is what you're going to see us implement today with Keycard.
08:46And I'm going to pull up this slide again.
08:49So, again, I'm going to basically demo or Mason and I are going to demo everything step-by-step on screen.
08:55Uh you are free to follow along or if you would prefer to just watch, you can also do that.
09:01Uh the workshop is going to use a pre-provisioned shared Keycard organization.
09:07So, if you do participate, you are going to be in the same organization as everybody else in the room.
09:13So, you'll be able to see what everybody else is doing.
09:17Okay, so if you go to the link that is that was provided, you're going to see this.
09:23This is the uh workshop running in essentially a sandbox.
09:31So, we're going to go ahead and start this.
09:32This is going to take a few min a minute or two to to uh get everything all set up.
09:40But, basically, kind of the premise here is we're going to start with an expense desk agent that does not require a login.
09:47It doesn't know who is acting at any point and everybody who uses the agent is using an API key.
09:56So, I have some folks uh here who are going to be helping to populate the demo, essentially.
10:04Um if you want to follow along as well, you would are going to eventually need uh admin access in key cards.
10:14So, as we go through the workshop, when we hit that part, uh they'll be watching to see people show up uh in the user list, and they will be granting you admin access as that happens.
10:28So, when you begin the workshop, this is what you're going to see.
10:30So, basically, this is the application where we don't know anybody's identity.
10:35Everybody is using the same MCP server, but they're all running their own agents.
10:40They are all using an API key.
10:44They might be using the same one, they might be using different ones.
10:46We, you know, don't really have a way of knowing that.
10:49But, if we look at the expense desk, it's we kind of see your your classical uh expense.
10:56You file an expense, and then it either gets approved or rejected, um and then paid out, which is not actually going to happen, so you're not going to receive money.
11:06Um But, basically, everything in this expense desk operates through the agent.
11:14So, the first thing we are going to do is we're going to submit an expense.
11:23So, I'm going to tell the agent I want to submit a $75 expense for snacks for the workshop.
11:29It submits that expense, and then its status is pending.
11:38Now, if we go down to the activity here, we can see that uh this expense was created using access granted by an API key.
11:48Right?
11:48So, we don't have any user identity, we don't know the agent's identity, the agent doesn't really have an identity yet on its own.
11:59And what I'm going to do is I'm going to refresh this.
12:04And I'm going to approve or deny somebody else's expense.
12:10So, let's see this one.
12:13If I hit approve, it's basically uh just going to send that as a prompt to the agent.
12:21It went ahead and approved that expense, and now we have the created by was created by this API key.
12:29So, I don't know who created that expense, right?
12:32Everybody's just using API keys.
12:34I approved it, but we also don't have any record of who approved it, either, because again, all we have is an API key.
12:51So, the next thing that we want to do is actually give the agent an identity, so that it can act on its own behalf.
13:05Uh so, the first thing we are going to do now is we are going to join that shared Keycard organization.
13:13So, if you click the link, if you're following along, if you click the link in the instructions, you will be able to you will get a prompt to log in with GitHub, and you can sign up with GitHub
13:26to join Keycard.
13:28When you get in, you should say see this workshop AI NYC in the upper left corner, indicating that you're in the right uh organization, and let me move this around so we can see things better.
13:53Okay.
13:53So, if you're following along, when you join, you're likely to see a couple of consent prompts.
13:59These are what I kind of talked about earlier in the presentation, where you are delegating some subset of your access to the agent to use on your behalf.
14:17So, this is the key card console.
14:19The first thing I want to talk about is applications.
14:22Now, applications and resources are registrations.
14:26So, something can be both an application and a resource.
14:29And basically, what this means is applications are things that are going to be able to make requests for tokens.
14:38They're going to ask for token exchanges in order to then exchange on down the line to access other resources.
14:46So, our The first thing we want to do is we want to create an application registration for our agent.
14:55So, what we are going to do here is I'm going to call this agent app.
15:09And then in order because we're all in the same uh organization here, we want to name things so that we can find our own applications and resources that we created.
15:18So, I'm going to use the my GitHub handle.
15:24Every app's identifier needs to be unique.
15:26So, we are going to do something similar here.
15:32We're going to use implicit consent.
15:36And then we need to grab our redirect URL.
15:39Now, this is something that we can get out of our uh sandbox image.
15:45So, if we go into this.
15:50I don't know why that one won't come to the front.
15:58Go to the terminal.
16:00We are going to run this.
16:04This gives you the redirect URL that is specific to your instance of the sandbox.
16:10So, I'm just going to copy that.
16:15I'm going to Oops, don't need to add this as a redirect URL.
16:24So, after there's a login event, this is where the user is going to be redirected back to.
16:32And I'm going to say create application.
16:42So, now we have a registration that says our expense desk agent can request tokens, right?
16:50So, but our expense desk agent can also receive tokens because it's going to receive the subject token from the user when the user logs in.
16:59And in order to do that, we basically need to do another registration, and we need to say that this agent application can also act as a resource, which means it is a target recipient
17:10for tokens.
17:12So, the authentication token is the token that this agent is going to be receiving.
17:18So, if we go into resources, we can add another resource.
17:22We are going to call this one The first one was agent app, this one is agent resource.
17:30I'm going to use my GitHub name.
17:34Let's use a similar identifier.
17:37This is the agent resource.
17:44I'm going to set the credential provider to zone provider, which essentially means that Keycloak is the entity that's going to be handling the token exchange and issuing uh tokens for this resource.
18:05And then I'm going to come down here, and I'm going to say this resource is being provided by my application, the agent app came out of.
18:15So, basically, this is the relationship that tells Keycloak that uh this application can exchange sign-in tokens addressed to this resource.
18:24If we don't make this association, then the exchange requests are going to be rejected.
18:33So, I'm going to save that.
18:38And now I want to add the things that my agent application depends on.
18:44So, if I come back to applications, I find my application.
18:52I'm going to go into dependencies, and then I'm going to look at the list of dependencies, and I'm going to add something here.
18:58So, the first one I'm going to add is the agent resource, and we need this because our agent application is going to request token exchanges to access uh downstream resources.
19:11We also want to be able to request token exchanges for the expense MCP server.
19:18So, this is a resource that's already been preconfigured for you.
19:20Everybody is using the same one, so we don't need to create a bunch of them that are all named after ourselves.
19:29Additionally, we have an agent, and we want to have the API key for that agent stored in Keycard so that we don't have its secret on disk.
19:38So, Keycard also provides a vault, and we have that API key configured in the vault already for you.
19:46So, we can select the LLM API.
19:51And finally, we're going to select this user info endpoint as well, because this is going to be able to tell us who the logged-in user is.
19:59It's going to be able to provide us with that identity information.
20:04So, I'm going to say connect.
20:11Now, the next thing I need to do is I need to give this agent app an its own identity, and I can't really do that unless it has some way to identify itself to essentially log in.
20:24Uh so, I'm going to click on add credential.
20:26I'm going to click on client ID and secret.
20:34Now, wait.
20:42We're going to go into the editor.
20:44We are going into the agent folder.
20:48We're going to find the end file.
20:51And we are going to paste in that client ID and that client secret.
21:01This is the secret.
21:04Now, we're doing this because this is essentially like a sandbox local environment.
21:09If you were using this in a production environment, you would probably use work something like workload identity.
21:15Also, Keycard provides a CLI which can populate secrets for you dynamically and not require them to be on disk.
21:25And then finally, I'm going to tell it what the identifier for the agent resource is.
21:30So, that was my your an agent resource and then my GitHub name.
21:37Going to wait until this says changes saved.
21:47And at this point, we have configured Keycard to be able to handle the token exchanges and exchange requests.
21:53Now, what we need to do is make sure that our expense desk application can actually handle that.
22:01So, essentially, when we go and look at the code here, we find that in the agent folder, we have this agent off.
22:12If we go down, we find Okay, this thing is in the way.
22:19I'm trying to move that.
22:28This we're going to find the checkpoint, which So, this this is the file where we don't have any authenticated MCP access, right?
22:41Now, the way the workshop works is if you at each lesson we have a checkpoint that will provide the code essentially for the end of that lesson.
22:49So, what we can do now is we can go and look for the checkpoint for exercise two, which is the one that we're at.
22:55So, if we go to checkpoints, we go to authenticated, and then we find the same file.
23:14Where are we here?
23:25Okay.
23:25So, we want to find this MCP tool set.
23:27This is passing in the keycard zone token off from the keycard SDK.
23:34And this is what is going to actually make that token request every time the agent actually wants to make a tool call with the MCP server.
23:44So, for every individual tool call, we're going to get a token that's scoped to that specific tool call.
24:01Now, this call is the this is Okay.
24:20So, now if we go to the keycard file, we look for this mint function.
24:31This is what actually goes ahead and makes that request to mint a token for the exchange.
24:41So, if we don't have a subject token, that is if we don't have a logged in user, then the agent is actually going to authenticate as itself using a client credentials grant.
24:52So, it's going to use that uh client ID and client secret to identify itself to Keycloak.
24:59And this means it's acting on its own behalf.
25:08Okay, so now if we go to the terminal tab, we basically want to make sure that we activate this updated code.
25:18So, we're going to uh run that so it switches over to checkpoint two.
25:24And then we have to restart the services.
25:30Stop.
25:31There it goes.
25:34Okay, now if we go and look at the expense desk, we have the screen that will let us continue with an authenticated expense desk agent.
25:49So, if we go ahead and click continue as the agent, then we're going to ask it uh which I can just copy this.
25:58Which account is being used right now?
26:02And what is my approval limit?
26:06And the approval limit is essentially the amount of money that you as a user are allowed to approve for other people's expenses.
26:15So, it says I'm signed in as the expense desk agent K Mida, which was in Keycloak that was the uh specific application that we are acting as that we provided the client ID and secret for.
26:30And the approval limit for the agent is about $50.
26:37So next I am going to tell the agent file a $75 expense or let's just say for some swag for the workshop on my behalf.
27:03So the agent submitted that expense.
27:08The memo says it was it filed on behalf of Kim but the actual activity details indicate that this was filed by this specific agent.
27:22So the on behalf of Kim is just something I told it.
27:25It doesn't necessarily actually have any to verify that because I as a user have not signed in it created this expense under its own identity.
27:50Okay.
27:50So now if we go into our applications actually I probably could have left that up.
27:57Oh, it's right there.
28:00We can see the activity in Keycloak.
28:02This is essentially the audit log that Keycloak generates of all the activities that have been happening against this application.
28:11And I am going to look for specifically because there's uh several different resources here.
28:31To look for the expense MCP resource.
28:36Anything that was calling the expense MCP resource, I'm going to limit this to the last hour.
28:51And we can see that there was a credential issued using the client credentials grant, which means that the actor was the application itself, and it was accessing the MCP resource.
29:08So now I'm going to go and look for what we have already registered for you, which is the expense MCP actor.
29:17So the expense MCP server has multiple registrations as well because it can receive tokens uh from the agent application, and then it can exchange that token on for a downstream token to access the ledger API.
29:38So if we click on this one, now we have uh because everybody is using the same resource for the MCP server, we're going to go ahead and just filter it down
30:00to it calling the ledger API.
30:09And I could technically use any of these, but oops, this is looking for people.
30:28So if we look at one of the calls that the expense MCP server used executed in order to exchange the token that it received from the agent application to access the ledger API.
30:43We can see that the delegation chain shows that the application that made the request was the expense MCP server and that it was acting on behalf of this application.
30:56And the resource that is accessing is the ledger API.
31:01So, this is essentially the the record of the delegation chain for the agent acting as itself.
31:10I'm going to go ahead and check that.
31:22Okay.
31:23So, now we have an agent that can identify itself and act as itself, but we maybe want it to act on our behalf but without impersonating us.
31:35So, we don't want it to just say Kim did this.
31:38We want it to say this agent did this thing on behalf of Kim with Kim's permissions.
31:44So, the next thing we're going to do is we're going to add human user identity into this process.
31:52So, now we have user delegation.
31:54At this point, we need to load the expense desk in a new tab because the iframe that's embedded into the sandbox doesn't handle external authentication.
32:07Um so, we're going to go ahead and open that in external window.
32:15Put this down here.
32:23And now we have this sign in button up on the right.
32:26I'm going to click that.
32:29If it's spinning.
32:36Okay.
32:37So, because I was already logged in, it went ahead and logged me in automatically.
32:42Um but now what I can do is I can say we can see that my email is here.
32:49If I ask the agent which account am I using now, before it said you're use you're acting as this agent, now it knows that I have an identity because I authenticated as a human user.
33:03All right.
33:04And my approval limit, which is associated with my identity metadata, is $100.
33:19So, I haven't filed any expenses under my human user yet.
33:23So, if I change this filter to to show all expenses, you can see that these expenses from earlier still um say that they were submitted by the API key.
33:35Like, adding all of this doesn't go back and change anything that's happened in the past.
33:40And we can see, you know, examples of the Expense Desk agent acting on behalf of other people as well.
33:51So, we have the ability now to act as ourself as a human user, but we have still retained the ability for the agent to act as itself as well.
34:01And the way that kind of plays out in this workshop is by the agent having the capability approve expenses that are under a certain amount of money.
34:16So, if I I'm just going to do this one.
34:23If I submit an expense for, say, $30 for office supplies for the team, it's going to verify who I am, it's going to file the expense.
34:47And then because the agent actually has the ability to approve expenses under $49.99 on its own, it filed the expense and then the agent approved it as itself.
35:01So if I click on the activity to examine what happened here, it says that this expense was created by me under my email.
35:11And then it was approved by the expense desk agent.
35:17So it shows, you know, that this is the identifier for the agent that acted in the approval and it gives a reason says eligible office supplies under $50 with a clear business purpose.
35:30Uh so now maybe I want to Let's do this $75 one as well.
35:47So if I submit an expense that is over that approval limit that the agent can act on on its own, then the status for that one is pending.
35:56A human needs to review it.
35:59If I go and say, "Hey uh I think this should be approved.
36:04So I'm going to send the message to the agent that says, 'Hey, approve expense 123.'"
36:16It says, "I can't approve or reject my own expense.
36:19So I have to ask another reviewer to do that."
36:22And it's able to do that because we have identities now.
36:25We know who all of the users are, who are logged in, and we also know the identity of the agent itself.
36:33Uh so before we didn't demonstrate this, but before I could have approved
36:37[music]
36:37my own expenses or anybody else's because it didn't know who anybody was.
36:43So if I go here, let's see.
36:53Let's find somebody else's pending expense.
37:01I'm going to say, uh, I want to reject this one.
37:24The agent goes through that song and dance.
37:26And now if we look at the recorded activity for this expense, it says that it was created by the expense desk agent, uh, because it was from kind of the previous cycle.
37:38And it was rejected by my identity.
37:44So, now we have a kind of a clear record of who filed what, who approved or rejected what.
37:56So, if we go into the KeyCloak console at this point, go to applications, we are going to find the Kitematic application.
38:09I'm going to clear any previous filters.
38:11I think they're already gone.
38:14I'm going to say actor, and I'm going to look for my email.
38:21Change this to hour.
38:26And if I click on one of these issued credentials, I can see that the application that, uh, made the token exchange request was my application agent app Kitematic, and it was acting on behalf of me, the human user
38:42Kitematic KeyCloak or Kitematic KeyCloak, and it was accessing the expense MCP resource.
38:50So, that was the first hop, right?
38:52So, now if we go over to the applications again and we want to find the expense MCP actor.
39:12Actually, I don't think we need to filter anything.
39:14It's all here.
39:16And we go to a credential that was issued for access to the ledger API.
39:20We can now see that on the second hop, the expense MCP server made the request on behalf of me and it was accessing the ledger API.
39:33So, on the first hop, what happened is the agent took my sign-in token that it acquired when I logged in as my human user and it exchanged the it made a token exchange request to keycard
39:48offering that sign-in token in exchange for a token for the next thing in in the uh chain, right?
39:55Which is the MCP server.
39:57So, it exchanged the subject token for an MCP server token.
40:01It then took that MCP server token and called the MCP server to make tool call with the appropriate tool scoped scopes.
40:09And then the MCP server took that token that it had received from the agent and it made a token exchange request to keycard asking to give that token up for
40:21a token to access the ledger API, which was the next resource in the stream.
40:30And you can trace these hops.
40:35You can trace these things across hops because they have a session associated with them.
40:40So, we're not going to spend a whole lot of time like digging into the the session numbers and copying them, but you can follow any specific sequence of events
40:50uh by the session number will be consistent on the first hop and also the second hop.
41:08I can close all of that.
41:10And I will now give it up to Mason.
41:14Awesome.
41:15Thank you so much, Kim.
41:16I have my little plushy tardigrade Ziggy goes with me everywhere.
41:19We have plenty of them.
41:20So, make sure you grab one before you leave.
41:22Take one home for your kids or for yourself.
41:24No shame.
41:25Whatever you'd like to do.
41:26So, we won't just walk through how to build all of this security and stuff into our agent and stuff.
41:31But now, how do we deploy these agents to ensure that they execute to their completion, that they don't fail or when they do fail, we know how to handle that, and we know how to
41:41reliably bring them back up so we don't lose all of our process and waste a whole bunch of time and compute.
41:45And that's where Temporal comes into play.
41:48[clears throat]
41:49So, Temporal is a durable execution framework.
41:52So, essentially what it does is we take your code, we execute it, we maintain the state of the application.
41:57If the application were to fail, run into an error, things of that nature, it will reconstruct the state or rebuild the state to the point of failure and continue executing
42:05onwards.
42:06Certain things such as function calls and things if say a distributed system were to go down, there are built-in retries and all sorts of failure handling modes.
42:14So, one of the things that we've seen a lot of with the customers that I work with and with our users is that we need a way of making
42:22your agents durable.
42:23And in reality, the agent loop is or the agent itself is just a loop.
42:27It's a it's a loop of ever calling tool calls to external services.
42:31Agents are just distributed systems at the end of the day with a little bit of added spice on top with some thinking.
42:36But the problems that we've solved for distributed systems apply to agent loops, and this is where Temporal comes in.
42:43It works out really well to keep your application running and going so you don't lose data, so you don't have to re- call LLMs, uh tokens and all of that nature.
42:51So, you So, your users don't experience any sort of outage.
42:54All sorts of really awesome things with that.
42:56So, what we're going to do a little bit is just kind of talk about a little bit of how you would structure uh Temporal application code.
43:01I'll leave this here for you to read later.
43:03I'm not going to dive super deep into the code itself, but essentially Temporal is a a system where you can you build a workflow and you define it by things what are called activities.
43:11Think parent function and child function calls.
43:14Easiest way to think about it.
43:15We implement our loop uh as a this parent function call and this becomes a workflow.
43:20And every single uh instance of or execution of this uh workflow, so like every single time we would have keycard go through and uh authenticate for user, that would be a workflow.
43:28And every single tool call is essentially an activity.
43:31These are things that can fail, that we expect to fail, and when they do fail, we either retry them or we rebuild the state and see what happens with it.
43:37So, we have our workflow calls here.
43:39I'm also I do the other direction for Mac Mac mouse.
43:42So, if I go the wrong direction, it's cuz I'm not not using my laptop.
43:46Um I remap everything on my computer.
43:49Uh and then down here you actually see like this async def SS.
43:51This is actually uh what an activity would look like.
43:53So, you it looks like normal Python code.
43:55It looks like the typical code you would write.
43:57Temporal is a code-first system, so you can build all of your things and hook it into anything that you would want.
44:02Um so we have our settlement code workflow, which I'm not going to read through too much.
44:06I'll leave that to you for you.
44:07But again, it's a Temporal workflow that uh essentially allows us to set how often we want to retry, and then we execute the activities.
44:15We have our um our debit ledger right here and our mark settled.
44:20So, we're able to, you know, add something to the ledger and then mark it as settled.
44:23And we have this timer here in the middle for us to kind of simulate some like uh human-in-the-loop time that it would wait take for someone to do this.
44:30Uh and then one of our activities uh for this, as you can see here, is uh this section right here, which is just the typical code.
44:37But I want to draw your line your eye to the to the first decorator on top, which is grant uh resource.
44:42So, Temporal exposes a lot of native uh primitives within its code base, uh one of which is what's called an interceptor.
44:48So, it allows you at certain points within the code to allow for essentially think of them as hooks.
44:53So, this is actually something that the Keycard team uh implemented, which is where whenever this um this activity is being executed, it actually goes and gets permission and gets an identity from Keycard,
45:05and then comes back, and essentially now we are able to authenticate this through uh our Temporal workflow and our activities through this, and we can maintain this identity separation that Kim was talking about, where you can have each individual worker have its own one.
45:16You can have a fleet of them have their own, but you'd be able to identify and limit scope, access, all of those wonderful things uh through there.
45:23Um so, the next thing we're going to do is we're basically going to find uh our work I think we already did the workers' identity in Keycard.
45:31Yeah.
45:32Okay, so we're going to start our worker.
45:33So, we actually already set up our Temporal application and set up all of these uh environment variable configurations.
45:38So, let's go ahead and start uh our worker.
45:40So, Temporal is a worker-server model a version of execution.
45:43You There's a Temporal service that maintains the execution, the application state, all of that.
45:48And then there's a fleet of workers.
45:49You can think Kubernetes pods, processes within a pod that your work as it's being uh as it gets as the request comes in.
45:56So, from here we'll go to start worker, and we'll just start our first worker right here, and so, we have worker PID 572 running.
46:04So, now we have a uh automatic uh a Temporal worker that's running the agent that was just done in the previous thing in the previous exercise, which will automatically
46:12uh accept our uh expenses.
46:14So, we have our worker running right here, and we see our PID is 572.
46:18And then we can start a settlement.
46:20So, what we're going to do is we're going to click on this client tab here, and we're going to invoke a settlement uh that's going to happen, and we'll see our workflow ID settlement came up.
46:28So, if we go back to our worker tab here, we'll actually see the output of our Temporal application as it's logging out what it is doing, and it's currently
46:36doing it as you can see here, we debited a token um and minted it for this particular call, and then we're running it.
46:42If we go to our Temporal Web UI, and we refresh this here, we can actually see our application running.
46:48I have to do this a little quickly, and then I can continue talking.
46:51Oh, no, it's not control Is it There it is.
46:53Okay, command-control-C you Macs is not fun for me.
46:56So, we had our application running.
46:58We said we had our debit ledger.
47:00I killed the application cold.
47:02So, as you can see here, the running state of this application is gone.
47:05We no longer have it.
47:06In a traditional system, this would be potentially problematic.
47:09We would have to you know, either have saved that state to a database and then re- uh rehydrate it when it comes back online.
47:16But, uh with Temporal, that's not really a problem because to the state of our application is stored right here inside of it.
47:22So, the very first uh the debit ledger uh function stored the output right here.
47:27So, we have a retry policy uh activity task completed.
47:29We debited 4,200.
47:31So, we have that done.
47:32There's nothing running right now, so it's going to start complaining and telling me, "Hey, you have nothing running.
47:36Our workflow task has timed out."
47:38We have a dead application.
47:40Uh and then we go back to our worker mid-settlement.
47:42Whenever we're ready, we can just restart the worker.
47:47And the application picks up as if the if the failure never happened.
47:51It rebuilt the entire state of our agent from all of the previous responses.
47:55No LLM calls were redone.
47:56No previous activities or functions were re-executed.
47:59It has picked up the state, replayed the actions, and then pushed it through, and we got the execution to work.
48:07So, why does durability matter for agents?
48:09Well, uh your agents are loops.
48:11Um there's a lot of things inside of your agents right now that cost a lot of money.
48:15Making LLM calls on on a massive scale is not cheap.
48:18There's their tokens you have to deal with.
48:20There's a lot of failure states.
48:21There's a lot of advanced states.
48:22Say you get through a really long loop and you know, you're 12 steps in and you lose the entire state of that application.
48:27Now you either have you've either had to built basically built a version of Temporal yourself where you are maintaining the state of your application in a database somewhere, and then you have to build all the reconstruction
48:36logic and all the retry logic, or uh you can use something like a durable execution system like Temporal, and it will do all of this for you.
48:43So, this allows for you to have your agents go offline, fail, your LLM provider can go down, you can lose an entire region in AWS.
48:51Doesn't matter when they when those when that compute comes back when those resources come back online, the agents pick up right where they left off at the exact last spot and continue execution.
48:59And if it happens quickly enough, if it's a quick enough failure, your user will never even be aware that they that they lost service of this.
49:05Um if you had been on the receiving end of this worker call that I had just done where we waited a minute and you were knew you were going to wait a minute for the approval and that failure happened and it happened and I restarted the worker within,
49:15you know, 5 seconds of the failure.
49:17As a user on the client side, you probably never would even notice that there was this kind of failure and that's what Temporal does.
49:22Is it brings it all the way out uh for you and just keeps the execution going forward regardless.
49:28Now, the interesting thing is um the secrets for all of this were actually not inside of our Temporal event history.
49:35So, if we go back to our client over here and we run this command, oop, I hit a button.
49:40Oh, I have to actually put in the correct workflow ID.
49:42If I don't, it doesn't work.
49:44If I go back here and collect this workflow ID.
49:49Uh it's on the client tab.
49:51Yeah.
49:52And we click the up arrow.
49:56Now, we're going to hope I know how to paste on Mac.
49:59Aha, we got it.
50:00Good.
50:01So, we can actually see we actually have a little little Python script here that says that no JWT token uh material uh was basically sent.
50:07We didn't like our worker through the entire event history here, um you can see the the starting process and the result.
50:14So, if we click in, I can see the debit ledger here with the result down here.
50:20So, we have our uh each individual function is stored there.
50:23So, our second function here, mark settled, was also uh settled as well.
50:27So, this was the result of it, but nowhere in our workflow if I go to the actual event history, which is this is actually what the event history looks like under the hood,
50:35you will not find it anywhere uh because we didn't store it.
50:38Um so, Temporal gives you basically two different ways uh to keep the sensitive data out.
50:43So, we have the interceptor, which is what Keycard provided for us.
50:45So, they built an interceptor that basically injects it at uh uh keeps the data and the credentials out of it entirely and then injects it back in when it's time to execute.
50:53Think of it like almost like just-in-time credentials uh and does that.
50:56Or the other thing for Temporal that's really awesome is there's what's called a payload uh encoder.
51:01So, what you can do is you saw in our Temporal Web UI, your data was There we go.
51:08Wrong button.
51:09Your data is just sitting here in nice wonderful plain text.
51:12You have the ability to encrypt this yourself.
51:15So, you can basically spin up There's like a There's an entire pattern and library in Temporal.
51:19You spin up a sidecar, you create an encryption algorithm, you set what the encryption policy is, and all data that comes to Temporal, whether it's self-hosted or in the cloud, is
51:29essentially encrypted.
51:30And we can't read it.
51:31We don't control your keys.
51:32We honestly don't quite care about what you're doing inside of it.
51:34That's not the part that we care about.
51:35We care about putting all these fancy things in order.
51:38So, that's a quick overview of like Temporal and building out it building out agents with it.
51:43There's a lot of integrations with Temporal you can try out.
51:45There's an Open AI, Pydantic AI.
51:49I know there's LangChain and a whole bunch of other ones too as well.
51:52Um And I think we even now have like an agentic harness now that we've been building out that's going out too.
51:57So, you can come by and see us at our booth if you want to learn more about that.
52:01Um And yeah, so the expense desk, you know, basically we I'll bring him back up to kind of wrap it up and we can take some questions if you want.
52:09Building out the agent identity, building out all of the agent worker and all of that stuff, and then Temporal becomes the execution layer to make sure that your agent runs properly
52:19and doesn't fail in the case of, you know, any sort of cataclysmic failure, API outage, AWS region just decides it wants to disappear for a day, which does happen.
52:27So, yeah, that's all I have.