{ "video": { "id": "Zf_e45GiNCE", "title": "Zero Cold Starts: Serverless AI Agents on Akamai Functions — Thorsten Hans", "duration": 1357, "upload_date": null, "channel": "AI Engineer", "source": "AI Engineer", "event_format": "" }, "analysis": { "video_id": "Zf_e45GiNCE", "title": "Zero Cold Starts: Serverless AI Agents on Akamai Functions — Thorsten Hans", "one_liner": "Compile MCP servers and AI agents to WebAssembly with the CNCF Spin framework, then run `spin aka deploy` to put them on Akamai Functions: they're distributed globally in about 60 seconds and cold-start on every request in under half a millisecond on average.", "summary": "Thorsten Hans argues that region-bound container and VM setups fail AI agents and MCP servers in three ways: users far from the region get high latency, cold starts take seconds to minutes, and overprovisioning to avoid them wastes money. Higher-level platforms that fix this tend to lock you in with proprietary SDKs. His alternative is server-side WebAssembly, which he calls the 'edge native AI stack': the vendor-neutral CNCF project Spin plus Akamai Functions as a globally distributed runtime. He demos scaffolding an MCP server with wasmcp, deploying a Rust decision-log MCP server backed by the built-in key-value store and using it from the Zed editor, and deploying a TypeScript game agent built with the Vercel AI SDK that calls an Ollama model on a Linode GPU instance.", "key_points": [ "Akamai Functions cold-starts every application on each request that hits its endpoint, in under half a millisecond on average. It is fully managed, with 'no knobs to turn, no terraform to write', and global distribution comes at no extra cost.", "Spin has three pillars: a CLI for the inner dev loop, a runtime based on Wasmtime (which also runs on microcontrollers and other constrained environments), and language-specific SDKs for HTTP, databases and similar. It is language-agnostic: anything that compiles to WebAssembly/WASI works.", "wasmcp is a CLI that plugs into Spin so you only write an MCP server's tools, resources and prompts. It handles non-functional concerns such as OAuth 2 / OIDC authentication by composing ready-made components into the final server (`wasmcp new` → `make` → `wasmcp compose` → `spin up -f server.wasm`). It has Rust, TypeScript and Python templates.", "The key-value store needs no configuration. Locally, Spin backs it with SQLite automatically. On Akamai Functions it uses the multi-tenant, globally distributed store, with no credentials to roll.", "How `spin aka deploy` works: the Wasm file and app manifest are packaged as OCI layers and uploaded. Every service region confirms it can run the workload, instantiates it once and runs a health check. Once all regions return HTTP 200, a generic subdomain is issued. The whole process takes roughly 60 seconds.", "When Spin compiles JavaScript, it loads the interpreter and your code, evaluates the global scope, and saves a memory snapshot into the Wasm file. Hans claims this makes JS-on-Wasm faster than native JavaScript on both cold starts and runtime performance. The trade-off is shipping the StarlingMonkey JS runtime, about 10 MB.", "In the agent demo, response headers showed traffic entering Akamai's network at a Santa Clara point of presence and being served from the LAX region. A full loop through the LLM took about 820–866 ms per x-envoy-upstream-service-time. An earlier 33–34 ms reading came back as an internal server error, which Hans put down to user error.", "Security comes from the WebAssembly sandbox: a workload cannot reach external resources unless the developer explicitly grants permission." ], "takeaways": [ "Before hand-rolling the MCP protocol over HTTP, try wasmcp. You write only the tools, with precise descriptions so humans and LLMs know how to use them, and compose in auth and transport components off the shelf.", "For agents, use any popular SDK inside a Spin TypeScript app. Hans suggests the Vercel AI SDK: define tools, give the agent an identity and instructions, and cap the number of steps.", "Chain compose and build steps in the Spin application manifest so `spin build` and `spin aka deploy` cover the whole workflow.", "Use Spin's built-in key-value store for persistence. You get SQLite locally and a globally distributed store in production with no code or config changes.", "Avoid lock-in by targeting open standards: Spin apps run on any WASI-compliant runtime, so Akamai Functions is just one deployment target." ], "topics": [ "mcp", "agents", "webassembly", "serverless", "edge computing", "cold starts", "developer experience" ], "tools": [ "Akamai Functions", "Spin", "CNCF", "WebAssembly", "WASI", "Wasmtime", "wasmcp", "MCP Inspector", "Zed", "Vercel AI SDK", "Ollama", "Linode", "StarlingMonkey", "Bytecode Alliance", "SQLite", "Akamai Property Manager", "Rust", "TypeScript", "Python", "Go", "OAuth 2.0", "OIDC", "Envoy", "Akamai Developer Hub" ], "quotes": [ { "text": "on average we can cold start in less than half a millisecond.", "at": "05:08", "url": "https://www.youtube.com/watch?v=Zf_e45GiNCE&t=308s" }, { "text": "So we are globally distributed in roughly 60 seconds.", "at": "14:12", "url": "https://www.youtube.com/watch?v=Zf_e45GiNCE&t=852s" }, { "text": "every web assembly application written in JavaScript is faster than a native JavaScript application when it comes to cold starts and runtime performance.", "at": "18:28", "url": "https://www.youtube.com/watch?v=Zf_e45GiNCE&t=1108s" }, { "text": "It's spin aka deploy is all you need.", "at": "21:39", "url": "https://www.youtube.com/watch?v=Zf_e45GiNCE&t=1299s" } ], "words": 3727 }, "summary_url": "/#Zf_e45GiNCE", "transcript": { "html": "/transcripts/Zf_e45GiNCE.html", "txt": "/transcripts/Zf_e45GiNCE.txt", "vtt": "/transcripts/Zf_e45GiNCE.vtt" } }