Why Are We Still Governing AI Agents Like Software? - Tim Tyler, Gravitee
https://www.youtube.com/watch?v=h8jgniNJcbQ
AI Engineer · 18 min · 231 sentences · sentence-level transcript from YouTube's caption track; each line begins at the time shown

[00:01] [music]
[00:13] Uh before we get started, uh my name's Tim and I'm with Gravity AI manage uh agent management.
[00:20] Uh before we get started, I'm sorry, somebody back in the office asked me to put this up real quick.
[00:24] So, bear with me.
[00:25] Uh we're hiring right now, so I'm going to put up a job description and if you know anybody it's in marketing operations.
[00:32] Um basically, no ID required.
[00:35] You can just come in no docs.
[00:37] Uh there's no manager or supervisor for this position.
[00:40] It's got a really obscure job description.
[00:44] You have access to key systems, but don't worry, they're all shared passwords.
[00:48] Uh and you will be expected to work 24/7, but nobody's checking on you and we won't fire you if you get anything wrong.
[00:56] I see some smiles and chuckles.
[00:59] Everybody raise your hand if you would hire somebody by this job description.
[01:03] See, nobody raised their hand, but we're already hiring this new digital workforce with AI agents.
[01:11] And if you aren't, then others are and they're probably your competition.
[01:15] So, today I'm going to talk about why we're still governing AI like software, where we look at it in as a at a different perspective of looking at it as a digital workforce.
[01:26] Of course, you need all of the software aspects.
[01:29] You need all of the protocols, the standards, so forth.
[01:33] But, they're not just necessary, we need something more.
[01:37] We need to look at it in a different way.
[01:39] I had somebody meet me at a show and said, "I'm I'm I'm head of AI at my organization and I sit between I go and talk to uh my CTO and they say, 'Well, it's not really a a software application.'"
[01:52] And he said, "Okay, so I went to HR."
[01:54] And they said, "Well, that's a software application."
[01:57] [gasps]
[01:57] So, we've done this before.
[01:59] This is how we started to look at this.
[02:02] Over the last century, we have been as enterprises, looking at how we govern the employee workforce, the humans.
[02:11] So, we said, "Well, what if we just take that and apply that onto AI agents and look at them as a digital workforce?"
[02:17] So, we call this agent accountability, and there's a framework behind this.
[02:20] I'm going to talk to you about the principles at the top of the framework, but there's controls for actually auditing to this framework as well.
[02:26] So, we break it down into three stages.
[02:29] We start with the assignment stage.
[02:31] This is what's essential to establishing the agent's presence in the organization.
[02:37] So, we start with who are you?
[02:40] So, if when you start with a company, you first they got to know who you are.
[02:44] You got to get into the registry into their system.
[02:46] And usually if it's like a secure organization, like many of you are in finance, you have a secure organization to get into the building, you have to start with a ID badge.
[02:57] And if your ID badge, like mine over at Verizon, if you're when I started with them and kind of get in the building, if your ID badge was like mine,
[03:06] it competed with your DMV photo for the worst picture of your life.
[03:10] So, we need to give the agent an identity.
[03:14] Who are you?
[03:16] That identity needs to be one unique ID.
[03:19] Needs to be in a known shared registry.
[03:22] That's one single point of truth.
[03:24] And that identity is what everything else in this framework is going to point to.
[03:29] Now, the agent doesn't just work alone.
[03:32] The agent is part of a team.
[03:34] It's a team, even if you're using an agent just between you and you're utilizing an agent, it is a team of two.
[03:41] And if it's part of a team for a specific role or task, or that is a team for um uh, larger team that's a department, or just the fact that this agent is in your organization is part of that team.
[03:56] So, it needs to get context on the goal, the decisions, the history of that team, and it should be giving context back into that team so the humans and other agents
[04:07] can know what it's doing and what what what is its learnings.
[04:12] And that means it's a teammate and not a digital clone.
[04:15] A lot of people think, "Oh, I'm just spinning up something here with Clyde and it's doing things on my behalf.
[04:20] Therefore, it is me."
[04:22] That's dangerous.
[04:23] So, the next step is we need to look at you've got your identity, but you need to look at who's the my boss, right?
[04:30] Who is responsible and accountable for what I do in the organization, and who's going to help me to understand what I'm supposed to do?
[04:37] So, we need to have the agent needs a sponsor who answers for you.
[04:43] So, our agent 409 that we've already given that identity to is now going to have an accountable party that is a named person.
[04:49] Sven is going to be the accountable party.
[04:53] Sven defines what agent 409's purpose and so forth.
[04:56] We'll get into that.
[04:57] He owns what 409 costs and what it spends, and he gets the call if anything goes wrong.
[05:05] So, there's one name that starts a line of accountability.
[05:10] That line of accountability we already have in our our hierarchy of our org chart, but the it starts with somebody where we enter that.
[05:19] The way I like to think of a line of accountability, or even the sponsor role, is when my kids were young, we had a pool and we had a rule
[05:28] that there was always somebody that was accountable for those kids in the pool, so that we'd know if they were drowning, right?
[05:34] And so, we had a rule that if I I was responsible for the kids, that if I needed to go to the bathroom or something, that I would look at some other adult that I trusted to be with the kids.
[05:45] I would look them in the eye, make eye contact, and say, "Do you have the kids?"
[05:49] And they would reply, "I have the kids."
[05:51] We did that for years.
[05:53] And that's essentially Sven is saying here, "I have the agent."
[05:58] So, now we've got our assignment.
[06:00] We know who and who is the agent and who who's responsible for it.
[06:04] So, now we need a line to what is this agent scope?
[06:07] What is it supposed to do and how is it supposed to do it?
[06:09] So, basically what are you supposed to do in this job?
[06:12] So, we've all had that job where you started and there was really no job description or really obscure job description and you just figured it out and kind of checked in with people and said, "Is this what I'm supposed to be doing?"
[06:21] Um that doesn't work for agents.
[06:23] They they are better at give them what are you here to do, a role.
[06:28] So, that job description is a purpose, responsibilities of how to accomplish that purpose, and some kind of budget.
[06:35] These are just high-level, of course a role goes way deeper than this.
[06:39] And that's what the controls on the framework help to define.
[06:42] But just to give an idea, what is the role?
[06:45] These are the types of things we're putting into it.
[06:47] And beyond what it's for, how to do the job, it's also what success looks like and we'll talk about that in a minute.
[06:56] So, there's two types of definition for the role.
[06:58] We've got onboarding versus runtime scope.
[07:01] So, a lot of what I just mentioned is that onboarding.
[07:04] What's the purpose and so forth.
[07:06] It's set once.
[07:07] It changes rarely unless we've got to take some recourse.
[07:10] Versus mission.
[07:12] And this is a word that I'm pulling from some of those text backs.
[07:15] Mission is like in the new AOTH.
[07:17] It's what's happening at this very moment.
[07:20] At every time it's making a call for instance to an MCP tool.
[07:23] So, the the agent actually defines its mission, but Sven always defines its role.
[07:32] So, we've got we've got a role, what we're supposed to do.
[07:35] But now we need a way to do it.
[07:37] So, we need to know what access and authorization we can have to to what data, to what tools to get the job done.
[07:45] So, we're going to give the agent authority.
[07:49] What can you see and what can you do?
[07:52] So, sponsor the sponsor Sven decides which doors.
[07:55] He's the one that gives the keys.
[07:57] Now, it might be through an IT department or a particular team that runs the MCP server, database, or whatnot, but it's Sven is the one that has to open it up for the agent.
[08:08] Cuz he's accountable.
[08:09] And this is where we start to set those boundaries and escalations are defined here on what how what happens with that authority.
[08:18] So, let me take a minute and talk about that authority in scope for a second.
[08:22] We always work to the lowest scope.
[08:24] What I mean by that, instead of going through specific scenario that could get confusing, I'm just going to call the access authority for Sven.
[08:32] He has access to A, B, C, and D. That could be information he's allowed to see, that could be a tool he's allowed to access.
[08:39] But he has that scope.
[08:40] Agent 203, notice because we have identities, I can call out this is not agent 409, this is agent 203.
[08:46] Agent 203's job is to converse with with Sven about projects.
[08:51] And agent 203 has a scope of C and D. And then agent 409 has D, E, and F. So, to get So, when agent 203 calls on behalf of Sven delegating a task,
[09:04] agent 203 calls to 409, 409 can actually go to D, E, and F to get the job done, but it can only return D back up the chain here to Sven.
[09:15] Because only D, Sven, 203, and 209 all have access to.
[09:21] So, the the scope of authority never grows beyond what agent 409 has been given.
[09:29] It always is constricted by what is that that lineage.
[09:34] Never assume alignment, meaning we have inherited uh alignment here with the model.
[09:41] For instance, if you are using a large language model built into that as its own alignments, right?
[09:47] It's own rules and its own definitions.
[09:50] As we know, the hyperscalers, the frontier models have just said they're misaligned.
[09:56] We're going to tell everybody they're misaligned.
[09:58] So, we know that we don't have control over that and that's locked in from our perspective.
[10:02] Also, that team context I was talking about is coming from somewhere else.
[10:06] So, we don't have Sven doesn't have the capability to go change that very easily.
[10:10] And then there might be outside agents that are accessing agent 409 from other parties.
[10:15] As we grow this whole agentic era, we're going to see this more and more.
[10:19] An outside agent kind of needs to be more like a contractor coming to work in your building.
[10:24] You give it an identity, you give it its authorization, what it's able to access, and so forth.
[10:28] But from the perspective of 409, we need to think that that that alignment, that context coming in is not something Sven controls.
[10:39] Opposed to Sven can control what is defined for 409.
[10:43] And so, we see he sets up a set of rules that are deterministic.
[10:46] That's the floor.
[10:47] That's kind of like FGA policies and so forth, where you can define specific things.
[10:53] But then you have intents that are probabilistic.
[10:55] So, he intends for something to get done.
[10:58] And how do we measure that?
[11:00] Oh, that's nice.
[11:02] And then uh so, the inherited alignment, it can change on us without our control even though we can't change it.
[11:09] And the defined alignment becomes a contract for 409.
[11:13] So, how do we actually enforce that contract?
[11:17] Well, that's with the administration stage here.
[11:19] So, with the administration is where we start to enforce everything before.
[11:23] So, if as an employee, we think we have trust, but also our enterprises or organizations make sure that they verify that trust.
[11:31] So, they have things like uh you know, human OS is over there rules that are making sure that our devices have a mobile device management on it that they're checking what we're doing and
[11:43] we have people checking in with us.
[11:44] We have supervisors that are watching that we're on a aligned to what we're working on.
[11:49] So, we need to look at that for agents as well, but one thing I want to call out here is that behaving that's what we talk about agents we say like are they out of scope?
[11:59] Are they accessing things they're not supposed to?
[12:01] That does not always equal succeeding.
[12:05] So, we need to think of behaving we defined in authority and succeeding we defined in the role.
[12:10] Are you actually succeeding at what we want you to do?
[12:14] So, the first thing we need to do in administration is we need to know if you're doing what you're supposed to do and you're staying within those those guardrails of authority.
[12:22] So, we set oversight.
[12:25] Are you doing what we expect?
[12:28] Now, that's not a single metric.
[12:29] It's actually a scorecard.
[12:31] So, on the authority side we have those more deterministic rules like are you staying within side the scope to authority?
[12:39] Are you you're only approved approved for up to $40,000 invoices to be dealing with something like that very clear rule and defined.
[12:49] But then on the role we tend to lean into things that are more that those intense.
[12:54] So, is your invoice match accuracy up to the par we want?
[12:58] Is your reconciliation time the cycle time within the scope?
[13:02] Yes, you can put rules to these, but more and more as we start to define these we start to look at them and say we we need to look at them as a scorecard.
[13:10] So, and the weights will come from the definitions in role and authority.
[13:15] So, again rules are checked and tense are judged.
[13:18] So, rules like I said these types of never approves a payment rights only to the ledger versus the intense are probabilistic and it's like the correct PO was matched a proper vendor
[13:28] project match.
[13:29] And an intent evaluation should be based on a confidence level.
[13:33] So if we look at that, well, who's actually looking at that?
[13:37] The agents need micromanagers.
[13:39] We as people, sorry, do not like micromanagers, right?
[13:43] We don't want that manager looking over our shoulder.
[13:45] But agents need that.
[13:47] So we have a micromanager looking at what's going on with the model.
[13:51] We have a manager looking at what's going on with the context coming into that agent.
[13:57] And then we also look at what's the work product coming out.
[14:00] And so there's all sorts of things we should be looking at in the oversight constantly.
[14:05] And that's why I have these as little guardian agents around here that are actually applying that at machine speed because if you wait for a quarterly review like we do with humans,
[14:14] you are about 40,000, 400,000, 4 million decisions too late to have that discussion.
[14:21] So, what happens when things start to go awry?
[14:24] We need to take action.
[14:26] Well, if you think about it, if we do something wrong, the first step isn't you're fired.
[14:30] Hopefully it's not.
[14:32] So you you're going to you're going to you know your supervisor is going to sit down and say, "Hey, you know what?
[14:36] You misunderstood that.
[14:37] This is what we want."
[14:38] A correction.
[14:40] But if you keep doing it, especially if you're out of your authority, we're going to start restricting that authority.
[14:45] And then if you keep doing it, we're going to escalate and then you're out of here.
[14:50] So we also need recourse.
[14:52] The same kind of model for agents.
[14:55] What happens if you don't do what we expect?
[15:00] So we look at it as a deterministic grade when we're talking about like intents for the probabilistic judgment, we look at like if if we're pretty confident, then we can take a certain action.
[15:13] If we're low confidence, we can take another.
[15:15] That allows us in the definitions to be to set these steps.
[15:19] And And steps allow actions.
[15:21] So for instance, if I'm looking at the intense and I've got a confidence score going, I can look at like if I'm 90% pretty confident, then I might just redact
[15:32] content if it if it's out of out of alignment.
[15:35] If it's less than 50%, I need to escalate to a human in the loop.
[15:39] If it's below 20%, block it.
[15:41] We're not We're not confident at all.
[15:43] So, by putting in those steps, those stops, we can start to turn that probabilistic into something that can have deterministic actions that can be acted on in recourse.
[15:53] But, that's opposed to something like you accessed something you're not supposed to access, and so we restrict authorization at the moment, or suspend access if you've done it five times today,
[16:05] or suspend the whole operation of the agent if it's been doing this consistently for days.
[16:11] So, that we when we look at a zero trust, we start heavier at first, then we lighten with repeated success.
[16:17] So, this is not something you just kind of set and then forget right away.
[16:21] It's something that you want to be looking at and evolving.
[16:24] And again, at the machine speed is where you can do this practically with agents.
[16:29] So, we have the police, the rule, we need a cop.
[16:33] That's where we've got something like guardian agents say gateways, guardrails, where they're doing restriction prohibit and suspension at machine speed.
[16:41] And then Sven's job, what we're good as at as humans, is we come in with our judgment to be able to say, "You know what?
[16:48] If I tweak this rule, if I tweak this authority definition in that alignment stage, then I can get a better result.
[16:56] I get more success out of the agent, or I get it to behave better."
[16:59] So, Sven adjusts those rules and authority as a coach, if you think about it that way.
[17:04] So, we have assignment, we need to name every agent.
[17:08] We have alignment, we need to bound every action.
[17:11] And we have administration, we need to trace every outcome.
[17:16] That's agent accountability.
[17:17] That's how we reach agent accountability.
[17:20] This was built from the history of humans, tailored for agents.
[17:25] Like I said, there's a a framework behind this with controls that I welcome you to take a look at that that you can audit in your organization, and then you can apply those technical solutions that I talked about as the methodology to achieve.
[17:39] So, you can get started today by giving your agents an identity.
[17:43] And one thing I ask when I give this talk is if everybody can do me a favor, give it the worst picture of his life.
[17:50] Thank you very much.
[17:51] I'm Tim Tyler in and uh feel free to go to agentaccountabilityframework.org or scan the QR code if you want my landing page, and you can check out the uh the agent accountability page or uh grab it if you want more information.
[18:03] We're also downstairs.
[18:04] Thank you.
[18:06] [applause]