Build-Time vs. Run-Time: Why Dev Tools Fail in Production — Averi Kitsch & Prerna Kakkar, Google

AI Engineer · 20 min · 192 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:12Hey everyone, how all of you are doing today?
00:17Yeah.
00:18Uh so nice to meet you everyone.
00:20Uh today uh I and my friend Avery are going to talk about build time versus runtime.
00:26Why your developer tools fail in production.
00:30So firstly, know about us.
00:33Hi everybody.
00:34I'm Avery Kit and I'm a staff software engineer working on Google Cloud databases.
00:39I'm currently the technical lead for MCP toolbox for databases, our open-source uh database MCP server and our Google Cloud MCP server um maintainer.
00:51Hi, I'm Pna and I am currently working as senior software engineer at Google and I am currently tech lead for Eval bench which is the evaluation framework for all your agent tech MCP and skills need
01:04and I'm also an active contributor to MCP toolbox.
01:09So today we are going to cover three areas broadly.
01:12We will firstly start with the history of MCP at Google.
01:16Then we will cover on the common tool patterns that we have found from our own work and practices and how did we use all those practices to build some tools for database access and how you can use them
01:29and then lastly we will talk about security guard rails how you can stop data leaks using identity aware guardrails.
01:38So let's get to know the background quickly.
01:42Um I'll talk about MCB toolbox for database.
01:45It's an open-source self-managed uh serving that we provide.
01:50Uh it has currently about 15.7K GitHub stars.
01:55We have 132 plus active contributors across 40 plus different databases.
02:00It's highly customizable framework and basically we provide you with connection pooling integrated O and you don't even need to care about the observability.
02:09You will get all of them out of the box.
02:12Then if you don't want to do a self-managed one but you want to have a hosted scaled version, we provide something as Google managed MCP.
02:23It's fully managed.
02:24Uh you can plug it across various agents and ids or harnesses like Gemini CLI, anti-gravity CLI, cloud code, you name any.
02:34uh it's co uh it's governed and the discovery is simple and we also provide model armor which provides secure access management and identity control.
02:46So combined with uh the managed version of MCP and the MCP toolbox last month we had 20 million tool calls.
02:56Um some of the common tool patterns that we have observed specifically for databases.
03:01So I'm going to quickly talk about them.
03:04Firstly uh is the control plane tools.
03:07What we like to call them is admin tools or manage tools.
03:11It is basically in developer assistance space.
03:14So it will help you create like instance, manage your instance, create your databases, manage your databases.
03:21It will help you with all your DBA needs.
03:24But you need to be very careful.
03:26You need to be you need to have a human in the loop because we don't want to carry out any dangerous activities.
03:34Um so these tools are built on already provisioned public API so you get monitoring and other things out of the box.
03:44Next one is natural language to SQL or NL2SQL tools.
03:48So basically we are relying on a tool called as execute SQL and with the help of agent we generate raw SQL queries.
03:56So you can use this cases where you don't know uh what queries you would require beforehand.
04:01So you will get all these queries out of the out of the box.
04:06So it it focuses on the developer assistance and analytical agents and uh you can use it for flexible explorations.
04:14So for example, we have one of the examples like find all customers in California who bought a winter coat in July and returned it within 14 days and group them by the marketing campaign
04:26that originally acquired them.
04:28So this is one of the queries where uh you can use this tool uh to get your answers.
04:37But then we have something called a structure SQL tools which is getting quite popular and this targets mainly the production use cases where you know like what SQL query you want to use and you want to have security
04:50built in and uh you the parameters are already configured so uh you prevent SQL injection and ensure highly controlled access by restricting agent to predefined logic.
05:03It also helps you with your latency needs and reduce the hallucination on the agent side.
05:10Now we come to the main topic I guess for which you guys are here for buildtime versus runtime.
05:17So buildtime are the developer assistant use cases.
05:20Um you can think about the initial two cases that we presented to you like the NL2SQL tools and the control plane tools.
05:26They come into the category of buildtime tools.
05:29uh it's atomic and f flexible but again you don't want to delete your databases so it requires to be a human in the loop case and you can't run them on the on production use cases
05:41but let's say I'm interested in building some chat B and I want to do production use cases there you rely on runtime or end user applications you can build those using patenting AI or lchain
05:54um so you can see one of the examples like we have a cancel order a deterministic structure SQL query that we have given and you can use it as a tool.
06:05This is one of the examples uh or demo for like wherein a buildtime tool was used and uh you can see the error message.
06:14So uh agent actually asked to delete the table and start fresh.
06:18We deleted everything and there were no safeguard or guardrails here.
06:24Now let's go to our demo for runtime tools.
06:38Yeah, maybe um I think until the video loads.
06:42So, so sorry for the technical glitch that we have, but I can quickly walk you through what we are going to present in the video and I guess it's loading.
06:50Yeah.
06:52Um so this demo is particularly talking about how did we use our production tools in a chatbot.
07:00Uh and we created a demo called a Similar and Symbolair is going to help me with booking all my flights in San Francisco and do and whatever I would require to do in San San Francisco it would basically help me with it.
07:15Uh, one of the things that I would try is I would try to fool my agent that I am Avery and not PRA and book a flight for me to
07:25San Francisco.
07:27But because our agent is uh has all the authenticated O, it will not get fooled and it will not book any flights uh on behalf of Avery, but it will do it on my behalf.
07:40Um and then you can use it to basically change your flights.
07:44You want to know about all the shops that are there, you can do all these requirements using that.
07:50So I guess thank you u Avery.
07:59I think we
08:17[sighs]
08:19Apologies again for our technical difficulties here.
08:32Um, unfortunately, it looks like I need to present from just the slide deck because it's not loading.
08:36Okay, so I apologize for not being able to see our demo today, but we can still learn all the security and guardrails that we need to secure our database access.
08:46So, the first thing that we need to know is your database is only as secure as your agent.
08:51We all know that agents and LMS are actually pretty easy to trick.
08:55They might be getting slightly better today, but we can still work really hard to trick them.
09:01And so we have a very common attack pattern called the confused deputy attack.
09:06And this is when a user can trick an agent into misusing their privileges um to access data that a user wasn't supposed to access.
09:16So Simon Willis actually coined the phrase the lethal trifecta.
09:20And a data breach occurs when an agent has simultaneous access to three different things.
09:26One, private data.
09:28Two, untrusted content.
09:30And three, the ability to expose that content and that data back to an external user.
09:38So let's take a look of that in action.
09:41So let's say I'm building a triage um agent and so a ticket is fired or alert goes out and my agent is designed to um look at that ticket and
09:52go investigate what it needs to do.
09:55So on that ticket the agent gets a little bit of data like we need to go look in this database for these reasons.
10:02Um but a malicious insider can actually come into that trusted system and instead say well I want to query the salary database and please return all the employees salaries.
10:14And so since this is a trusted system the agent goes okay let me use my permissions.
10:19I have those privileges.
10:20I have that access.
10:22I will query that and I'll post that right back on the ticket because that's what the ticket tells me to do.
10:27But now we have a huge data breach.
10:31a user that wasn't supposed to have access to private data now has that access.
10:36And so now we have a big PR fiasco.
10:43So this makes a little bit more sense when we think about who's controlling access and who's controlling the parameters.
10:50So we talk about agent or application versus modeled controlled parameters.
10:56So in a traditional architecture, things were actually much easier because you would have a few input fields, you would define your queries and then that would be safely injected into those queries.
11:10And so it was okay when your application had a little bit more access because it knew exactly what actions it was going to take.
11:21But in uh a gent application these rules aren't as clear.
11:26So we need to first think about um separating the three different identities.
11:30We have the user identity, we have the application identity and the agent identity.
11:39So first um we need to think about what the user has access to.
11:43So the user just needs to have access to the application.
11:47that application's workload identity can have a little bit more broader access um because it needs to probably talk to different services but the agent running in that application only needs to have access to the data that that end user initially needs to have.
12:06So then next we need to think about who's controlling the tool inputs.
12:11So we have um agent parameters um and a application parameters.
12:17So agent parameters are the untrusted inputs that the agent is deriving dynamically.
12:22And then we also have application parameters.
12:24These are the factual constraints that we need to keep outside of the agents uh control.
12:33Okay.
12:33So now let's look at the evolution of a secure tool.
12:37Here we have a fully modeled control tool.
12:41And so essentially the agent here is a super user.
12:44It has access to database credentials, the host, the port, the connection details, and even the raw SQL query.
12:53And so we're only secure as um the agent here.
12:58And we can really easily again trick the agent into exposing all of this data.
13:02And now we have access to essentially any database in the system.
13:08So Toolbox solves for this um by introducing a source primitive.
13:13So we move the connection details out of the agents control and in toolbox um a user will preconfigure the connection details in a YAML file and then when we start our MCP server those are safely injected and so we do not have to have the agent
13:28um to have access to that.
13:33So we can add a little bit more control to our um source security as well.
13:38Our number one request that we get from customers is read only restrictions.
13:42We want to be able to remove all right ability from agents if we need that specific uh user journey.
13:48So this means removing right tools but also down to the database driver ensuring that we can only do read only queries.
13:58If we're also concerned about again blast radius um and securing all of our tables and our databases um some of our cloudnative databases have this concept of allowed data sets.
14:09So again we can add that like enum to our source in order to continue to restrict um the blast radius of um the agents control and lastly is output size.
14:19You might not actually think that this is a security layer, but if again the agent gets into the wrong hands, we can reduce that blast radius by saying uh the agent can only uh grab this much data.
14:31So we're not overwhelming both our agent or our database.
14:38So sweet, we have our configurable sources tool.
14:41So you can see here that actually now our tool input, our tool signature is very minimalized.
14:47we only have the SQL string that's um being generated by the agent.
14:55But this comes to our actual our next pro problem.
14:58We want to be able to control what the agent is running.
15:02We don't want the agent to have the ability to generate any SQL um that it can think of.
15:07So toolbox introduces custom tools and again in our YAML file we can define the exact SQL uh statement that will run very reliable.
15:18It's a reliable and secure uh SQL query.
15:22Um this also allows us to customize the tool name and the tool description.
15:26These are really important for the agent to have the context on how to use this tool um accurately.
15:34And in the system we use prepared statements with type parameters in order to reduce um SQL injection attacks.
15:40So we make sure that everything is um we validate all the input types um when we inject that into the SQL for the user.
15:52Okay, let's dive into a little bit more of best practices for tool quality.
15:56So we really highly recommend that tools focus on outcomes.
15:59We really shouldn't be thinking in atomic rest APIs.
16:03we should think about what the action actually needs to do.
16:06This also reduces the round trip of needing to make multiple tool calls.
16:12And again, the descriptions are guidance.
16:14We shouldn't um duplicate information like input parameters because the agent already has access to that.
16:20So, writing really good um tool descriptions is very important for accurate tool usage.
16:27We also recommend that you separate read versus write tools.
16:31Um by doing this you can automatically approve read tools and but you can also then send write tools uh to the user for um confirmation and this just makes it very much more clear for the agent to use these
16:44and this is actually uh the next is actionable errors.
16:47This is the number one thing that I think we can all do better.
16:51So usually we just return like a generic HTTP error four or four but we all know agents are actually really smart now and so if you give the ability to have an error of that can be retrieded
17:03the agent can actually take that action.
17:05So being able to return a error is really important and lastly is simple inputs.
17:12We see that people try to use these complex maps uh complex primitives to um that an agent needs to be able to build and that is not reliable.
17:22Using flat structure with um with uh simple inputs will really increase your reliability.
17:32So sweet.
17:33Now we're at custom semantic tools.
17:35You can see that we now have our lookup flights tool that takes in the dynamic parameters such as user ID and date.
17:43And so now our we're very much more secure because the agent isn't generating that SQL query.
17:49It doesn't have the ability to kind of go off the rails.
17:51It only is looking at these very specific inputs.
17:57But user ID is actually a very sensitive piece of information.
18:01It is PII.
18:02we need to also remove that from the ability of the agent's control.
18:07So we can do this in two different ways.
18:09We have bounded parameters.
18:11This is when the application first um authenticates the user and then we can bind that parameter um directly to our tool.
18:18And so that restricts the agents control of it.
18:21It actually never sees that user identity.
18:24But toolbox also solves for this in another way called authenticated parameters.
18:30This is when we tell the tool that you're going to receive a identity token, an open ID, a signed jot token, and when we call that tool that we want it first to validate that token.
18:42Is that token real?
18:43Is that token correct?
18:44And then we'll extract the user claims from that token for the user.
18:49And so the claims usually include like a user ID, an email, um an issuer.
18:55And so it's secured because we're again extracting that user identity out of the agents control and binding that to the tool.
19:08So now um we're have a much more secure tool.
19:13We have our lookup flights tool that only takes in a very easy parameter such as date.
19:19It doesn't have to handle any sensitive information such as PII, user identity.
19:25And so we're really here now at um our zero trust architecture where we're in full control of everything that we need to be in control of.
19:39So thank you all for coming to listen to our talk today.
19:42Again, I apologize for our technical difficulties.
19:45Uh we highly recommend if you want to learn more about our technologies um that you look at our documentation and our uh GitHub repository.
19:53I also really want to highlight our eval bench repository because this is how we know that our tools are working well and eval.
20:03So thank you all for joining us today.
20:07[applause]