How Secrets Leak Through AI Agents (and How to Stop It) — Venice AI

AI Engineer · 15 min · 114 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:12Hi everyone.
00:13Uh, welcome to my talk.
00:14I'm Joshua Mo and today we're going to be talking about some of the challenges of building a privacy respecting AI system.
00:26So just for an introduction as to who I am, I'm currently the lead developer relations engineer at Venice AI.
00:34So currently I handle partnerships, documentation, talking to developers, um some engineering and other things other things as well.
00:48Uh so prior to this I was actually the lead maintainer for rig which is the leading AI framework in Rust.
00:58Um so what is Venice?
01:02So Venice is a private and unrestricted AI platform.
01:06Uh so we host all the latest open-source models.
01:11We're 100% open AI compatible.
01:15We don't store any prompts or conversations and we also have a web app that you can use to talk to our models as well.
01:26We also have a studio offering that is um a offering for our content creators that includes video, audio, image generation and transcription.
01:42So why does privacy matter?
01:45So let's d deep dive into this.
01:48So as we know data breaches are increasingly becoming an eventuality which means that essentially reputational risk matters.
02:00You only need one data breach before your users stop trusting you and they move on to another provider.
02:07Or if you're enterprise, maybe you lose the enterprise customer who's been fueling almost 90% of your annual revenue.
02:18So in this case, sensitive data can severely increase the impact of any security incident.
02:25As well as that, storing less use uh user data directly reduces the blast radius of a security security incident that happens.
02:37So, how does Venice do privacy today?
02:40Well, there's a few ways.
02:42The first way is that we don't store any prompt conversations, of course, as we mentioned previously, which means that any digital footprint incurred in your side is is essentially reduced to zero.
02:54So, when you use models like GLM 5.2 and Kimmy 2.7, you can guarantee that none of your conversations will be saved on our side.
03:03It will be all stored locally.
03:05We also have predefined tiers of privacy.
03:08So private by default means that the affforementioned uh measures apply for providers like open AI and claude.
03:20Uh we anonymize your user footprint so that none of well they can't identify you from your simply from your prompt or your request.
03:32We also have models hosted in trusted execution environments, meaning that you can verifiably prove that your inference was run in a totally private process.
03:46As well as that, we also use a privacy first software architecture.
03:52So for endpoints like video uh generation where you have to pull the endpoints, we store the results in a totally encrypted payload using split key custody, meaning that attackers can't access your data and neither can the team should their accounts be compromised.
04:12So I just want to introduce you to the guys the uh the idea that not all systems are made equal.
04:18So, not all systems are made private.
04:22Some companies may claim their systems are private, but they're not really.
04:26Uh, you guys know how it goes.
04:29So, while models are a very important part of an aentic system, in fact, they're a vital part of the system.
04:38Um, they are still only one part.
04:41And in fact to create a complete system to accompany an LLM in doing a task you have to build an entire process around it.
04:51However, these systems are also much more prone to data leakage and breaches.
04:56So the real question is how can we optimize for this kind of problem.
05:02So one example I just wanted to bring up was observability logging.
05:06So observability as many of you know is a core part of any serious production system today.
05:13So just to break it down quickly, observability can be broken down into metrics, logs and traces.
05:20So traces are simply a span of work uh done over a unit of time.
05:25Metrics as are simply numbers like counters, gauges for instance and logs are the actual messages.
05:32So logging, as we all know, is actually pretty great for debugging, especially in production or dem dev environments.
05:40But you can also accidentally store prompts and secrets by default if you're not careful, which can also hold personal information.
05:50However, when you use Venice, you don't need to worry about any of that.
05:56So let's talk about how secrets can flow through a regular agent.
06:00So for example, let's say you're you're using open claw.
06:04You send a user request that gets sent to an agent runtime.
06:08The runtime might need a secret that it will fetch from a one password vault for instance or hashc vault.
06:16The secret will then get sent to the LLM reasoning layer.
06:24Um, oh my god.
06:31So the reasoning layer will then make an API request through a tool call which will then return the answer back to the user.
06:42However, due to this the way that it's architected, the secrets get fed directly to the LLM, which means it then may get used in training data.
06:52So in terms of privacy preserving patterns that work, one thing that we've used at Venice is storing hashes that rep represent ids that we need as opposed to direct data.
07:06So for example, if you need to identify a user for rate limiting purposes, you might not want to use their direct user footprint directly.
07:15You might want to, you know, store it as a hash for instance, just so you can identify them without directly using any of their personal data.
07:24As well as that, we encrypt data and store it in an inaccessible enclave using split key custody.
07:31So for context, encryption keys are stored in two halves, a read half and a write half.
07:38By splitting these, the attacker cannot access the data due to needing both halves for the decryption to complete.
07:49Additionally, you can use design patterns like handles that can be easily revoked.
07:54So a handle is simply an object that grants you access to something.
08:00So for example, let's say you have a database pool using a handle.
08:05So when you have the handle, you can access the database pool.
08:09When you don't, then you can't access it.
08:13This is very important especially when you're using authorization like JSON web tokens or session keys that may be uh a core part of your authorization system.
08:26As well as that, of course, engineering your prompts with privacy in mind can be uh very helpful for engineering teams who are using coding harnesses like cursor and clawed code
08:40in terms of protecting agent handle data.
08:42So essentially ensuring narrow agent scope to to decrease risk.
08:48So for example, let's say you pull tools from an NCP server that may have 15 20 different tools.
08:55you don't really want to use all of them.
08:56You actually only really need one, but you also need an API key to handle that uh API call.
09:05So if you pull that API key in from you know your your password vault and then you send it as part of the agent um what can happen is that it gets exfiltrated
09:18through through the agent uh on a tool hall to whatever you know web server is being hosted by a bad actor for instance.
09:27As well as that um we also have a classification layer in front of an agent to check for prompt injection for example.
09:37So there's a couple of ways you can do this.
09:40So you can either use an LLM based uh layer to do this or you can alternatively use a semantic um search layer for example using vector search uh via something like quadrant
09:57lance db or additionally you can use an in-memory process.
10:05It should also be noted that tight context loops caused by narrow agent scope can also improve context window performance.
10:14So for example, because you don't have as many tools, uh your agents may be able to perform much better.
10:21It's well known that when you fill your context window, by doing so, when you reach near the limit of your context window, performance can actually degrade uh considerably.
10:35By keeping an empty or near empty context window, we can make sure that the agent is performing at its best.
10:46So let's look at keeping secrets out of the model.
10:51So here we have a user request that gets sent to the reasoning layer.
10:56This reasoning layer is actually then uh formulating a series of non-deterministic inputs for an agentic runtime.
11:06So by tying the password vault to the agentic runtime instead of the reasoning layer, what happens then is that the run the agent runtime is essentially turned into an execution
11:18layer which then we can you know we can consider it deterministic.
11:23We can do all kinds of things that you normally wouldn't be able to do for privacy and security reasons at the LLN layer.
11:33Uh so for example, let's say you want to use a uh browser user tool and you need to log into a into a website that requires your credentials.
11:45With this uh software architecture, it's much more reasonable to say that it's private and secure rather than feeding your password see uh directly into the reasoning layer as well as you know calling external APIs.
12:06So in terms of sandboxing your agents um it it's a very well-known fact that using sandboxes and VMs for example firecracker um and you know AWS EC2 in a VPC
12:20can significantly upgrade the security of your agent.
12:26However this doesn't act as a cure all for your agentic systems.
12:32Uh so for example, if you have internet access enabled for your agent, it can still accidentally exfiltrate data through external API access if you're not careful and therefore monitoring permissions
12:46is essential.
12:49So my personal suggestion is building your own agentic runtime.
12:57So essentially building your own coding harnesses where you can add your own steps, your own observability, uh your own tools, things of that nature.
13:07Essentially owning the execution layer and reasoning layer.
13:13So let's talk about privacy at the hardware level very quickly.
13:16So in terms of this um there's something I like to call confidential compute where a trusted execution environment essentially uh at the hardware level enables the inference process to be run privately.
13:30So with Venice, how this works is that when you send a request, uh you get a response and then we actually return an attestation that you can then um
13:43verify with our atestation verifier.
13:47So in terms this what this actually means is that you can fully isolate an agent at the reasoning layer cryptographically and prove it in terms of oh confidential tool calls there is still some work to be done um tees
14:03are still great for tool calls however we also offer end to end encryption um but there's also you know some work to be done there because of uh the need to provide an ergonomic
14:16API I experience while still allowing users to return entirely encrypted tool calls.
14:23This is still kind of an ongoing challenge.
14:26Um in terms of this, a very reliable solution would be to allow providers to own the computes layer at which your tool calls being run.
14:35However, um
14:37[laughter]
14:40sorry, just give me a sec.
14:51So in terms of confidential tool calls um yeah there's definitely some work to be done.
15:02Um yeah that's it.
15:03Thank you for listening.