Build-Time vs. Run-Time: Why Dev Tools Fail in Production — Averi Kitsch & Prerna Kakkar, Google
https://www.youtube.com/watch?v=9R--1tg45Jg
AI Engineer · 20 min · 192 sentences · sentence-level transcript from YouTube's caption track; each line begins at the time shown

[00:01] [music]
[00:12] Hey everyone, how all of you are doing today?
[00:17] Yeah.
[00:18] Uh so nice to meet you everyone.
[00:20] Uh today uh I and my friend Avery are going to talk about build time versus runtime.
[00:26] Why your developer tools fail in production.
[00:30] So firstly, know about us.
[00:33] Hi everybody.
[00:34] I'm Avery Kit and I'm a staff software engineer working on Google Cloud databases.
[00:39] I'm currently the technical lead for MCP toolbox for databases, our open-source uh database MCP server and our Google Cloud MCP server um maintainer.
[00:51] Hi, I'm Pna and I am currently working as senior software engineer at Google and I am currently tech lead for Eval bench which is the evaluation framework for all your agent tech MCP and skills need
[01:04] and I'm also an active contributor to MCP toolbox.
[01:09] So today we are going to cover three areas broadly.
[01:12] We will firstly start with the history of MCP at Google.
[01:16] Then we will cover on the common tool patterns that we have found from our own work and practices and how did we use all those practices to build some tools for database access and how you can use them
[01:29] and then lastly we will talk about security guard rails how you can stop data leaks using identity aware guardrails.
[01:38] So let's get to know the background quickly.
[01:42] Um I'll talk about MCB toolbox for database.
[01:45] It's an open-source self-managed uh serving that we provide.
[01:50] Uh it has currently about 15.7K GitHub stars.
[01:55] We have 132 plus active contributors across 40 plus different databases.
[02:00] It's highly customizable framework and basically we provide you with connection pooling integrated O and you don't even need to care about the observability.
[02:09] You will get all of them out of the box.
[02:12] Then if you don't want to do a self-managed one but you want to have a hosted scaled version, we provide something as Google managed MCP.
[02:23] It's fully managed.
[02:24] Uh you can plug it across various agents and ids or harnesses like Gemini CLI, anti-gravity CLI, cloud code, you name any.
[02:34] uh it's co uh it's governed and the discovery is simple and we also provide model armor which provides secure access management and identity control.
[02:46] So combined with uh the managed version of MCP and the MCP toolbox last month we had 20 million tool calls.
[02:56] Um some of the common tool patterns that we have observed specifically for databases.
[03:01] So I'm going to quickly talk about them.
[03:04] Firstly uh is the control plane tools.
[03:07] What we like to call them is admin tools or manage tools.
[03:11] It is basically in developer assistance space.
[03:14] So it will help you create like instance, manage your instance, create your databases, manage your databases.
[03:21] It will help you with all your DBA needs.
[03:24] But you need to be very careful.
[03:26] You need to be you need to have a human in the loop because we don't want to carry out any dangerous activities.
[03:34] Um so these tools are built on already provisioned public API so you get monitoring and other things out of the box.
[03:44] Next one is natural language to SQL or NL2SQL tools.
[03:48] So basically we are relying on a tool called as execute SQL and with the help of agent we generate raw SQL queries.
[03:56] So you can use this cases where you don't know uh what queries you would require beforehand.
[04:01] So you will get all these queries out of the out of the box.
[04:06] So it it focuses on the developer assistance and analytical agents and uh you can use it for flexible explorations.
[04:14] So for example, we have one of the examples like find all customers in California who bought a winter coat in July and returned it within 14 days and group them by the marketing campaign
[04:26] that originally acquired them.
[04:28] So this is one of the queries where uh you can use this tool uh to get your answers.
[04:37] But then we have something called a structure SQL tools which is getting quite popular and this targets mainly the production use cases where you know like what SQL query you want to use and you want to have security
[04:50] built in and uh you the parameters are already configured so uh you prevent SQL injection and ensure highly controlled access by restricting agent to predefined logic.
[05:03] It also helps you with your latency needs and reduce the hallucination on the agent side.
[05:10] Now we come to the main topic I guess for which you guys are here for buildtime versus runtime.
[05:17] So buildtime are the developer assistant use cases.
[05:20] Um you can think about the initial two cases that we presented to you like the NL2SQL tools and the control plane tools.
[05:26] They come into the category of buildtime tools.
[05:29] uh it's atomic and f flexible but again you don't want to delete your databases so it requires to be a human in the loop case and you can't run them on the on production use cases
[05:41] but let's say I'm interested in building some chat B and I want to do production use cases there you rely on runtime or end user applications you can build those using patenting AI or lchain
[05:54] um so you can see one of the examples like we have a cancel order a deterministic structure SQL query that we have given and you can use it as a tool.
[06:05] This is one of the examples uh or demo for like wherein a buildtime tool was used and uh you can see the error message.
[06:14] So uh agent actually asked to delete the table and start fresh.
[06:18] We deleted everything and there were no safeguard or guardrails here.
[06:24] Now let's go to our demo for runtime tools.
[06:38] Yeah, maybe um I think until the video loads.
[06:42] So, so sorry for the technical glitch that we have, but I can quickly walk you through what we are going to present in the video and I guess it's loading.
[06:50] Yeah.
[06:52] Um so this demo is particularly talking about how did we use our production tools in a chatbot.
[07:00] Uh and we created a demo called a Similar and Symbolair is going to help me with booking all my flights in San Francisco and do and whatever I would require to do in San San Francisco it would basically help me with it.
[07:15] Uh, one of the things that I would try is I would try to fool my agent that I am Avery and not PRA and book a flight for me to
[07:25] San Francisco.
[07:27] But because our agent is uh has all the authenticated O, it will not get fooled and it will not book any flights uh on behalf of Avery, but it will do it on my behalf.
[07:40] Um and then you can use it to basically change your flights.
[07:44] You want to know about all the shops that are there, you can do all these requirements using that.
[07:50] So I guess thank you u Avery.
[07:59] I think we
[08:17] [sighs]
[08:19] Apologies again for our technical difficulties here.
[08:32] Um, unfortunately, it looks like I need to present from just the slide deck because it's not loading.
[08:36] Okay, so I apologize for not being able to see our demo today, but we can still learn all the security and guardrails that we need to secure our database access.
[08:46] So, the first thing that we need to know is your database is only as secure as your agent.
[08:51] We all know that agents and LMS are actually pretty easy to trick.
[08:55] They might be getting slightly better today, but we can still work really hard to trick them.
[09:01] And so we have a very common attack pattern called the confused deputy attack.
[09:06] And this is when a user can trick an agent into misusing their privileges um to access data that a user wasn't supposed to access.
[09:16] So Simon Willis actually coined the phrase the lethal trifecta.
[09:20] And a data breach occurs when an agent has simultaneous access to three different things.
[09:26] One, private data.
[09:28] Two, untrusted content.
[09:30] And three, the ability to expose that content and that data back to an external user.
[09:38] So let's take a look of that in action.
[09:41] So let's say I'm building a triage um agent and so a ticket is fired or alert goes out and my agent is designed to um look at that ticket and
[09:52] go investigate what it needs to do.
[09:55] So on that ticket the agent gets a little bit of data like we need to go look in this database for these reasons.
[10:02] Um but a malicious insider can actually come into that trusted system and instead say well I want to query the salary database and please return all the employees salaries.
[10:14] And so since this is a trusted system the agent goes okay let me use my permissions.
[10:19] I have those privileges.
[10:20] I have that access.
[10:22] I will query that and I'll post that right back on the ticket because that's what the ticket tells me to do.
[10:27] But now we have a huge data breach.
[10:31] a user that wasn't supposed to have access to private data now has that access.
[10:36] And so now we have a big PR fiasco.
[10:43] So this makes a little bit more sense when we think about who's controlling access and who's controlling the parameters.
[10:50] So we talk about agent or application versus modeled controlled parameters.
[10:56] So in a traditional architecture, things were actually much easier because you would have a few input fields, you would define your queries and then that would be safely injected into those queries.
[11:10] And so it was okay when your application had a little bit more access because it knew exactly what actions it was going to take.
[11:21] But in uh a gent application these rules aren't as clear.
[11:26] So we need to first think about um separating the three different identities.
[11:30] We have the user identity, we have the application identity and the agent identity.
[11:39] So first um we need to think about what the user has access to.
[11:43] So the user just needs to have access to the application.
[11:47] that application's workload identity can have a little bit more broader access um because it needs to probably talk to different services but the agent running in that application only needs to have access to the data that that end user initially needs to have.
[12:06] So then next we need to think about who's controlling the tool inputs.
[12:11] So we have um agent parameters um and a application parameters.
[12:17] So agent parameters are the untrusted inputs that the agent is deriving dynamically.
[12:22] And then we also have application parameters.
[12:24] These are the factual constraints that we need to keep outside of the agents uh control.
[12:33] Okay.
[12:33] So now let's look at the evolution of a secure tool.
[12:37] Here we have a fully modeled control tool.
[12:41] And so essentially the agent here is a super user.
[12:44] It has access to database credentials, the host, the port, the connection details, and even the raw SQL query.
[12:53] And so we're only secure as um the agent here.
[12:58] And we can really easily again trick the agent into exposing all of this data.
[13:02] And now we have access to essentially any database in the system.
[13:08] So Toolbox solves for this um by introducing a source primitive.
[13:13] So we move the connection details out of the agents control and in toolbox um a user will preconfigure the connection details in a YAML file and then when we start our MCP server those are safely injected and so we do not have to have the agent
[13:28] um to have access to that.
[13:33] So we can add a little bit more control to our um source security as well.
[13:38] Our number one request that we get from customers is read only restrictions.
[13:42] We want to be able to remove all right ability from agents if we need that specific uh user journey.
[13:48] So this means removing right tools but also down to the database driver ensuring that we can only do read only queries.
[13:58] If we're also concerned about again blast radius um and securing all of our tables and our databases um some of our cloudnative databases have this concept of allowed data sets.
[14:09] So again we can add that like enum to our source in order to continue to restrict um the blast radius of um the agents control and lastly is output size.
[14:19] You might not actually think that this is a security layer, but if again the agent gets into the wrong hands, we can reduce that blast radius by saying uh the agent can only uh grab this much data.
[14:31] So we're not overwhelming both our agent or our database.
[14:38] So sweet, we have our configurable sources tool.
[14:41] So you can see here that actually now our tool input, our tool signature is very minimalized.
[14:47] we only have the SQL string that's um being generated by the agent.
[14:55] But this comes to our actual our next pro problem.
[14:58] We want to be able to control what the agent is running.
[15:02] We don't want the agent to have the ability to generate any SQL um that it can think of.
[15:07] So toolbox introduces custom tools and again in our YAML file we can define the exact SQL uh statement that will run very reliable.
[15:18] It's a reliable and secure uh SQL query.
[15:22] Um this also allows us to customize the tool name and the tool description.
[15:26] These are really important for the agent to have the context on how to use this tool um accurately.
[15:34] And in the system we use prepared statements with type parameters in order to reduce um SQL injection attacks.
[15:40] So we make sure that everything is um we validate all the input types um when we inject that into the SQL for the user.
[15:52] Okay, let's dive into a little bit more of best practices for tool quality.
[15:56] So we really highly recommend that tools focus on outcomes.
[15:59] We really shouldn't be thinking in atomic rest APIs.
[16:03] we should think about what the action actually needs to do.
[16:06] This also reduces the round trip of needing to make multiple tool calls.
[16:12] And again, the descriptions are guidance.
[16:14] We shouldn't um duplicate information like input parameters because the agent already has access to that.
[16:20] So, writing really good um tool descriptions is very important for accurate tool usage.
[16:27] We also recommend that you separate read versus write tools.
[16:31] Um by doing this you can automatically approve read tools and but you can also then send write tools uh to the user for um confirmation and this just makes it very much more clear for the agent to use these
[16:44] and this is actually uh the next is actionable errors.
[16:47] This is the number one thing that I think we can all do better.
[16:51] So usually we just return like a generic HTTP error four or four but we all know agents are actually really smart now and so if you give the ability to have an error of that can be retrieded
[17:03] the agent can actually take that action.
[17:05] So being able to return a error is really important and lastly is simple inputs.
[17:12] We see that people try to use these complex maps uh complex primitives to um that an agent needs to be able to build and that is not reliable.
[17:22] Using flat structure with um with uh simple inputs will really increase your reliability.
[17:32] So sweet.
[17:33] Now we're at custom semantic tools.
[17:35] You can see that we now have our lookup flights tool that takes in the dynamic parameters such as user ID and date.
[17:43] And so now our we're very much more secure because the agent isn't generating that SQL query.
[17:49] It doesn't have the ability to kind of go off the rails.
[17:51] It only is looking at these very specific inputs.
[17:57] But user ID is actually a very sensitive piece of information.
[18:01] It is PII.
[18:02] we need to also remove that from the ability of the agent's control.
[18:07] So we can do this in two different ways.
[18:09] We have bounded parameters.
[18:11] This is when the application first um authenticates the user and then we can bind that parameter um directly to our tool.
[18:18] And so that restricts the agents control of it.
[18:21] It actually never sees that user identity.
[18:24] But toolbox also solves for this in another way called authenticated parameters.
[18:30] This is when we tell the tool that you're going to receive a identity token, an open ID, a signed jot token, and when we call that tool that we want it first to validate that token.
[18:42] Is that token real?
[18:43] Is that token correct?
[18:44] And then we'll extract the user claims from that token for the user.
[18:49] And so the claims usually include like a user ID, an email, um an issuer.
[18:55] And so it's secured because we're again extracting that user identity out of the agents control and binding that to the tool.
[19:08] So now um we're have a much more secure tool.
[19:13] We have our lookup flights tool that only takes in a very easy parameter such as date.
[19:19] It doesn't have to handle any sensitive information such as PII, user identity.
[19:25] And so we're really here now at um our zero trust architecture where we're in full control of everything that we need to be in control of.
[19:39] So thank you all for coming to listen to our talk today.
[19:42] Again, I apologize for our technical difficulties.
[19:45] Uh we highly recommend if you want to learn more about our technologies um that you look at our documentation and our uh GitHub repository.
[19:53] I also really want to highlight our eval bench repository because this is how we know that our tools are working well and eval.
[20:03] So thank you all for joining us today.
[20:07] [applause]