{
  "video": {
    "id": "cI7zfqusmFU",
    "title": "Every step you take, every call you make: the reliable agent stack — Giselle van Dongen, Restate",
    "duration": 1250,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": "In person"
  },
  "analysis": {
    "video_id": "cI7zfqusmFU",
    "title": "Every step you take, every call you make: the reliable agent stack — Giselle van Dongen, Restate",
    "one_liner": "Giselle van Dongen argues that as agents become long-running, persistent, asynchronous entities, the missing layer is durable-execution infrastructure — and demos Restate running a Slack deep-research agent that survives crashes, suspends for human approval, and can be signalled or cancelled mid-loop.",
    "summary": "The talk skips the AI part entirely and focuses on the infrastructure layer needed to run agents resiliently in production. Van Dongen frames the shift using Karpathy's three waves — LLM-as-website, agent-as-app, and now persistent asynchronous entities living in your infrastructure — and argues agent SDKs and memory tools don't solve the distributed, stateful, long-running problem, leaving teams to hand-write retry and recovery logic. She demos Restate, an open-source durable-execution framework that sits in front of your agent service as a proxy, journals every event over an open connection, and replays that journal to recover a run exactly where it failed. Live demos cover a planner/parallel-researchers/writer deep research agent in Slack: recovering from an injected web-search API failure, suspending on a durable promise for human approval, signalling extra context into a running agent loop, and cancelling a run so the cancellation rewinds down the stack through sub-agents.",
    "key_points": [
      "Karpathy's three waves framing: LLM as a website you ask a question, then an app you download with tools, now persistent asynchronous long-running processes with access to tools, other agents and context — so the infra layer must evolve from single agents to agentic platforms.",
      "Restate's four ingredients: durable execution for a single agent run; thousands of concurrent isolated sessions with consistent state; communication between agents, MCP servers and tools; and control — cancelling or killing a stuck or misbehaving execution.",
      "Architecturally Restate is a separate server sitting in front of the agent service like a message broker or proxy; it pushes the request to the service and keeps an open connection that acts as a 'lifeline', collecting a journal of events used to recover after failure.",
      "Code model: ordinary HTTP handlers become durable via the SDK's restate object context; an inline litellm call is made durable just by wrapping it in `restate.run`, so a failure two hours or two months later resumes at exactly that point.",
      "Human approval is modelled as a durable promise — a suspension point in the journal — so the process suspends while waiting (consuming no serverless execution time) and survives restarts and redeploys over weeks.",
      "Virtual objects are stateful actors with a unique ID (e.g. session ID), isolated key-value state such as chat history, and handlers; they guarantee only one execution runs at a time per key, queueing a second behind the first so concurrent agents can't overwrite each other's session state.",
      "Live demo of interacting with a running agent: an LLM classifies a Slack follow-up as relevant ('focus on frontier models') and signals it into the ongoing loop, or irrelevant ('forget about that, research AI policy') and cancels — cancellation propagates down the call chain, killing sub-agents first, then the controller, rewinding the stack.",
      "Extensibility demo: when a hypothetical expensive new model blows up cost, the inline LLM step is pulled out into its own handler acting as an LLM gateway with a policy check; other agents call it via Restate's distributed communication primitives, which also give flow control (e.g. one department limited to 300 concurrent calls to the gateway).",
      "Internals: an event-driven distributed log inspired by Meta's core event infra, with an event loop that persists state in an embedded state store, sets timers, or sends requests to other agents; it's a push model rather than the pull model of most workflow orchestrators, giving ~45ms p99 for a 10-step workflow and working well with serverless wake-ups.",
      "Operationally it's a single binary including the state store and UI; run it highly available by spinning up multiple instances snapshotting to object storage. Six SDKs, integrations with popular agent frameworks, or bring any LLM SDK. Open source and self-hostable, plus BYOC (deployed in your cloud account, data never leaves) and managed cloud."
    ],
    "takeaways": [
      "Stop hand-rolling retry and recovery logic around agents — adopt durable execution so a run recovers from its journal at the exact failed step instead of restarting from the beginning.",
      "Model long-lived agents as stateful entities (virtual objects keyed by session ID with isolated state and single-writer concurrency), not as linear workflows, if users need to interact with an in-flight run.",
      "Implement human-in-the-loop approvals as durable promises that suspend the process, so waiting weeks for a click costs nothing on serverless and survives redeploys.",
      "Give every invocation an addressable identifier so other processes can fetch its output, signal new context into its running loop, or cancel it — and rely on cancellation propagating down through sub-agents.",
      "When cost or policy pressure arrives mid-project, factor shared LLM calls out of inline steps into a gateway handler with policy checks and per-team concurrency limits rather than deploying new infrastructure."
    ],
    "topics": [
      "agents",
      "durable execution",
      "infrastructure",
      "workflows",
      "human-in-the-loop",
      "distributed systems",
      "serverless",
      "observability"
    ],
    "tools": [
      "Restate",
      "Apache Flink",
      "Meta",
      "Slack",
      "litellm",
      "MCP",
      "Python"
    ],
    "quotes": [
      {
        "text": "Think about things like when an agent runs for a week and then crashes. We want to be able to bring it back and let it continue exactly at the point where it failed. We don't want it to start over from the beginning.",
        "at": "03:02",
        "url": "https://www.youtube.com/watch?v=cI7zfqusmFU&t=182s"
      },
      {
        "text": "So instead of starting over, it uses the journal to recover the progress.",
        "at": "06:52",
        "url": "https://www.youtube.com/watch?v=cI7zfqusmFU&t=412s"
      },
      {
        "text": "So a workflow is not the nicest way to model this kind of thing.",
        "at": "09:30",
        "url": "https://www.youtube.com/watch?v=cI7zfqusmFU&t=570s"
      },
      {
        "text": "latencies of for example 45 milliseconds p99 for like a 10-step workflow",
        "at": "18:41",
        "url": "https://www.youtube.com/watch?v=cI7zfqusmFU&t=1121s"
      }
    ],
    "words": 3857
  },
  "summary_url": "/#cI7zfqusmFU",
  "transcript": {
    "html": "/transcripts/cI7zfqusmFU.html",
    "txt": "/transcripts/cI7zfqusmFU.txt",
    "vtt": "/transcripts/cI7zfqusmFU.vtt"
  }
}