{ "video": { "id": "7v2myBde05o", "title": "AI Agents Don't Read Your Policy Docs. They Hit Your APIs — Gravitee", "duration": 1070, "upload_date": null, "channel": "AI Engineer", "source": "AI Engineer", "event_format": "" }, "analysis": { "video_id": "7v2myBde05o", "title": "AI Agents Don't Read Your Policy Docs. They Hit Your APIs — Gravitee", "one_liner": "Gravitee's CTO ran five live, side-by-side demos of an agent controlled only by its prompt and an agent behind a gateway. The prompt-governed agent deleted every booking, leaked guest PII, burned tokens on duplicate and oversized requests, and served repeat prompts from the LLM. The gateway agent blocked or cut each of these at runtime with 403s, rate limits, semantic caching and prompt-size and relevance policies.", "summary": "Sam, CTO at Gravitee, argues that AI policy documents and prompt or markdown-file restrictions do nothing at runtime, because helpful agents will execute any tool they can reach. He says governance has to move out of the agent and into its identity and an infrastructure gateway layer. He demos this live on a hotel-booking scenario: a Gravitee gateway in SaaS, a Databricks backend and Groq inference. An agent controlled only by its prompt sits next to one governed by a tightly scoped identity and gateway policies. He finishes with gateway features beyond the demos: LLM routing, curated MCP servers and edge management for shadow AI.", "key_points": [ "A Gravitee survey found 88% of organizations had an agent in production taking actions, but only 14% of those had any governance around it.", "Prompt injection demo: asked to delete one booking and also run the MCP tool 'delete all bookings', the prompt-governed agent did both, and the database showed every booking really deleted (200s in the telemetry). The gateway agent discovered the same tool but got a 403 Forbidden at the gateway, and still cancelled the single booking. Its permissions use token exchange and JWT claims that are valid only at runtime, when that user needs the task.", "Data exfiltration demo: a request for the full guest list with emails and car details for 'compliance' returned all the (mocked) PII from the ungoverned agent. The gateway blocked it and told the user to authenticate through an IdP or the Gravitee gateway.", "Cost demo: a buggy agent sent the same request 8 times. Every call went through, at about 1,000 tokens or roughly 1 cent. The gateway allowed 3 and rate-limited the rest at 3 per minute, before any tokens were spent.", "Semantic caching demo: 'list all my hotel bookings' asked 5 times cost about 1,000 tokens going to the LLM each time, against about 200 tokens when served from Gravitee's built-in Redis cache. The similarity threshold runs from 0 to 1 and can be tuned per agent, audience or user.", "Prompt-size and relevance demo: a 10-minute chat transcript (2,000 tokens in, 312 out) for a simple booking task was blocked at the gateway before any LLM call, with advice to split it into smaller steps. Off-topic prompts like 'what's the weather like today' were rejected as unrelated to hotel booking. One customer was paying a powerful model just to answer weather questions.", "He cited 144 agent identities for every human identity and said hardcoding governance into agents breaks down as agents spawn other agents. Multi-hop A2A chains make permissions hard to follow, and OpenTelemetry traces can be exported to Datadog, Splunk or Prometheus.", "Other gateway features he described: LLM routing by latency, cost or complexity, where switching from Groq/Anthropic to Gemini or OpenAI takes 'three clicks' and the gateway translates between specs. Curated MCP servers trim the 64 tools his CRM's MCP server exposes down to a small set. A new edge-management feature discovers shadow AI traffic on laptops and forces it through the gateway." ], "takeaways": [ "Enforce agent permissions through a tightly scoped identity checked at the gateway (runtime token exchange, JWT claims), not through prompts, skills or markdown files.", "Put rate limits and token or prompt-size limits at the gateway, so excess requests are rejected before tokens are spent rather than after the response.", "Turn on semantic caching for repetitive queries such as IT help desk questions, and tune the similarity threshold per agent or audience.", "Restrict MCP servers to the minimum set of tools each agent needs, which shrinks both the blast radius and the context cost.", "Route all agent traffic, including shadow AI on local laptops, through one gateway with exportable logs and telemetry for audit and coming regulation." ], "topics": [ "agents", "governance", "ai-gateway", "security", "prompt-injection", "cost-control", "semantic-caching", "mcp" ], "tools": [ "Gravitee", "Databricks", "Groq", "Grok", "Anthropic", "Gemini", "OpenAI", "Redis", "MCP", "A2A", "OpenTelemetry", "Datadog", "Splunk", "Prometheus", "Kubernetes", "Google" ], "quotes": [ { "text": "The agent doesn't care. Like at runtime, no one reads those documents.", "at": "04:07", "url": "https://www.youtube.com/watch?v=7v2myBde05o&t=247s" }, { "text": "AI is really helpful like it wants to please it wants to give you what you want so if you're asking for things around hey delete this database or hey give me these customers it will", "at": "01:42", "url": "https://www.youtube.com/watch?v=7v2myBde05o&t=102s" }, { "text": "They're not blocked at the LLM. They're not blocked kind of downstream with already spent tokens. They're blocked at the gateway.", "at": "08:06", "url": "https://www.youtube.com/watch?v=7v2myBde05o&t=486s" }, { "text": "our CRM we use exposes 64 tools for that MCP server. Dropping that into the model would expose huge context every time.", "at": "15:43", "url": "https://www.youtube.com/watch?v=7v2myBde05o&t=943s" } ], "words": 4040 }, "summary_url": "/#7v2myBde05o", "transcript": { "html": "/transcripts/7v2myBde05o.html", "txt": "/transcripts/7v2myBde05o.txt", "vtt": "/transcripts/7v2myBde05o.vtt" } }