{ "video": { "id": "_0SQh8a1Osk", "title": "Don't Vibe Code Your Agent Infrastructure - Akshat Bubna, Modal", "duration": 1055, "upload_date": null, "channel": "AI Engineer", "source": "AI Engineer", "event_format": "" }, "analysis": { "video_id": "_0SQh8a1Osk", "title": "Don't Vibe Code Your Agent Infrastructure - Akshat Bubna, Modal", "one_liner": "Modal's CTO argues that agents need purpose-built sandbox primitives (multi-container sandboxes, persistent snapshots, elastic CPU/GPU, fine-grained egress control and observability), and that you shouldn't 'vibe code' that infrastructure yourself because reliability, security and scale are too hard to get right.", "summary": "Akshat Bubna opens with Heavy AI's product, where every cell in a spreadsheet is served by its own agent running in a Modal sandbox. He uses it to list what agent infrastructure needs: per-agent workspaces, persistence that doesn't burn compute while idle, elastic resources, GPUs, shared high-performance file systems, tight network and credential control, and observability that agents can use themselves. He argues against building a sandbox platform 'based on intuition', for three reasons: reliability, security and scaling. He backs this with Modal's scale numbers. He closes by predicting that the harness ('brain') will be separated from the sandbox ('hands'). That would move agent infrastructure past 'pets versus cattle' to an 'ocean of microbes'.", "key_points": [ "Heavy AI built a product where you open a table and run a query, and literally every cell is served by a separate agent. Each agent runs in a Modal sandbox. Thousands of agents must spin up instantly and shut down when the query finishes.", "Modal sandboxes can now run multiple containers or microVMs inside one sandbox. This lets the control loop (model API calls, launching tools) run separately from where tool calls execute, for security and isolation. The pattern uses sidecars plus shared storage, e.g. agents in the same table row sharing memory.", "Agents need a full workspace. Ramp Inspect, Ramp's internal coding agent that 'seems to handle almost every PR in Ramp', needs the repo, dependencies and dev services. Basis, an accounting agent, needs documents, working papers and scripts. A game-dev customer needs a game engine and assets.", "Persistence has to survive indefinitely without burning compute, e.g. an agent waiting days for an accountant to reply. Modal offers several kinds: full root filesystem snapshots, single-directory snapshots (e.g. node_modules) and shared volumes.", "Ramp Inspect data shows huge variability in CPU and memory use, both between sandboxes and within one sandbox over time. Provisioning for the peak would mean about 15 CPU cores and 15 GB of RAM per sandbox, which is 'simply not economically feasible'. Modal allocates on demand and bills only for actual use.", "Agents increasingly need GPUs (perhaps later TPUs or AMD). 'Cloud Science' has a ready-made Modal integration that hands heavy CPU/GPU work to Modal sandboxes. Recursive self-improvement is a use case: agents given GPU sandboxes to optimize training swap GPUs and run ablations in parallel.", "Allowlists alone aren't enough for security. Users want path-level control (e.g. which DataDog or GitHub paths an agent can reach) and credentials injected so the agent can't see them. A pattern is emerging of users writing lightweight proxy sidecars with specific rules; Ramp does this.", "Modal scale figures: 20 million snapshot images per week, about 8 PB of data per day, about 2.6 million volumes per week. One personal-assistant customer needs 10,000 sandbox creations per second and almost a million concurrent sandboxes, each living 2–3 minutes and interactive in under a second. Modal pools capacity from multiple hyperscalers and neoclouds because of GPU and CPU shortages." ], "takeaways": [ "Don't build your own sandbox platform 'on intuition'. Reliability under failure, keeping security boundaries current, and scaling are hard even with AI help.", "Separate the agent control loop from tool execution with multi-container sandboxes and sidecars, and put a rules-based proxy in front of external services instead of relying only on domain or IP allowlists. Inject credentials so the agent never sees them.", "Don't provision sandboxes for peak load. Use elastic, pay-per-use resources and snapshot or persist state so idle agents waiting on humans cost nothing.", "Give agents themselves access to logs, metrics (e.g. GPU utilization) and budget data, so they can diagnose failed runs and iterate on their own.", "Design for harnesses that create, snapshot and coordinate many environments dynamically, rather than treating agent environments as uniform cattle." ], "topics": [ "agents", "sandboxes", "agent infrastructure", "persistence", "security", "observability", "scaling", "gpus" ], "tools": [ "Modal", "Heavy AI", "Ramp Inspect", "Ramp", "Basis", "GitHub", "DataDog", "Kubernetes", "Cloud Science" ], "quotes": [ { "text": "You can right now in their product open a table and run a query where literally every cell is served by a separate agent.", "at": "01:14", "url": "https://www.youtube.com/watch?v=_0SQh8a1Osk&t=74s" }, { "text": "The worst thing about agents is that they can make a mistake and not realize that they are transferring your confidential data, for example, to a German forum.", "at": "09:59", "url": "https://www.youtube.com/watch?v=_0SQh8a1Osk&t=599s" }, { "text": "So if you were to allocate resources for peak loads, you would have to provide about 15 CPU cores and 15 GB of RAM for each sandbox, which is simply not economically feasible.", "at": "07:11", "url": "https://www.youtube.com/watch?v=_0SQh8a1Osk&t=431s" }, { "text": "great primitives allow your agents to do more.", "at": "17:19", "url": "https://www.youtube.com/watch?v=_0SQh8a1Osk&t=1039s" } ], "words": 2813 }, "summary_url": "/#_0SQh8a1Osk", "transcript": { "html": "/transcripts/_0SQh8a1Osk.html", "txt": "/transcripts/_0SQh8a1Osk.txt", "vtt": "/transcripts/_0SQh8a1Osk.vtt" } }