{
  "video": {
    "id": "xo68uCibfm8",
    "title": "It’s Tokens All The Way Down: How RLMs are Different — Kevin Madura, AlixPartners",
    "duration": 1248,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer"
  },
  "analysis": {
    "video_id": "xo68uCibfm8",
    "title": "It’s Tokens All The Way Down: How RLMs are Different — Kevin Madura, AlixPartners",
    "one_liner": "RLMs (recursive language models) keep the context as a live variable in a Python REPL that the model manipulates with code and hands off to sub-LM calls, so you stop doing context engineering and just define inputs, outputs and intent.",
    "summary": "Kevin Madura of AlixPartners explains what a recursive language model is and why it differs from RAG, agents and tool calling: instead of strings passing back and forth, the context lives as an object in a symbolic environment (typically a Python REPL) that the model interacts with directly, and it can recursively delegate subtasks to another LM — often itself. Because the raw input never enters the main context window, only results that matter come back, sidestepping context rot; benchmarks like OOLONG and BrowseComp show RLMs beating GPT-5-with-BM25 tool calling at lower cost. He demos a cohort retention analysis over three data frames in ~20 lines of code, shows traces from the DSPy/RLM-native platform Compound, and surveys real uses: invoice consolidation at Trampoline AI, log analysis, harness optimization with Halo, and a security report over a 500,000-line OWASP vulnerable app. His framing is 'bitter lesson pilled': define the objective and the typed inputs/outputs, and defer everything in the middle to the model.",
    "key_points": [
      "An RLM has two defining properties: the context is an object it interacts with symbolically in a REPL environment (not a JSON string round-trip), and it can delegate to another LM — the same model or a different one — which recurses down.",
      "Omar's DSPy tweet on handling arbitrary-length inputs (summarizing an arbitrarily long document into a table of contents) was, to the speaker, the first inkling that context windows may not need deliberate management.",
      "On the OOLONG and BrowseComp long-context benchmarks the RLM line sits on top; the purple comparison line — GPT-5 doing tool calling against a BM25 tool — is more expensive for worse performance.",
      "RLMs dodge Dex's 'dumb zone' / context rot because the full input stays as a variable in the REPL and only results that actually matter re-enter the main model's context.",
      "Anthropic's recently released workflows are doing something similar — intermediate results live in script variables — and someone from Anthropic cited the RLM paper at the CIS conference as a key driver of that implementation.",
      "Raymond's testing on the long chain-of-thought benchmark showed a jump from 2.6% to 45.4% accuracy, with the strongest gains on code-amenable tasks: logic puzzles, chess, chemistry.",
      "Toy examples that break base models: summing 12 numbers buried across 30,000 tokens (trivial with a regex the model writes itself) and working over data frames, where the model edits them as if typing in its own Jupyter notebook.",
      "Running the same experiments through Claude Code was 'totally bloated' — for production workloads he argues for a structured pipeline with defined inputs and outputs rather than `claude -p` and hope.",
      "PredictRLM (Trampoline AI) uses DSPy to define the schemas between the main LM and sub-LM calls, which the speaker thinks buys readability, maintainability, and possibly better performance from cheaper models like Qwen.",
      "The model decides when to stop: you set a max-iterations variable, it explores until comfortable, then calls submit and returns the typed outputs you declared up front."
    ],
    "takeaways": [
      "Reach for an RLM when the input context is large or dense, the output is huge (hundreds of thousands of lines — an underexplored case), the task decomposes, or the session is long-horizon; skip it when the work fits in context, latency matters, or the model isn't a strong coder.",
      "Stop building chunking and embedding scaffolding for long documents like 200-page invoices or contracts — define the inputs, the output types and the guidance, and let the RLM churn through it.",
      "Wrap the RLM in a deterministic shell: specify inputs, outputs and intent, and let the model decide the implementation in the middle — the same discipline DSPy imposes.",
      "Enforce typed schemas on the main-LM-to-sub-LM handoff (as PredictRLM does with DSPy) so the delegation stays readable and the sub-model returns precisely the shape you asked for.",
      "Don't ship a coding agent as your production pipeline just because it works interactively — measure it against an RLM on cost and bloat, not just on whether the answer comes out right."
    ],
    "topics": [
      "rlm",
      "context-engineering",
      "dspy",
      "long-context",
      "agents",
      "benchmarks",
      "tool-calling",
      "recursion"
    ],
    "tools": [
      "AlixPartners",
      "DSPy",
      "PredictRLM",
      "fastRLM",
      "Ax",
      "Compound",
      "Anthropic workflows",
      "Claude Code",
      "GPT-5",
      "BM25",
      "GLM 5.2",
      "Qwen",
      "inference.net",
      "Trampoline AI",
      "Halo",
      "OOLONG",
      "BrowseComp",
      "OWASP vulnerable web app",
      "Jupyter",
      "AWS"
    ],
    "quotes": [
      {
        "text": "The key difference here is that it treats the context as an object that it can interact with symbolically in its environment.",
        "at": "00:40",
        "url": "https://www.youtube.com/watch?v=xo68uCibfm8&t=40s"
      },
      {
        "text": "It's actually interacting directly with the data frame as if it was typing in its own Jupyter notebook.",
        "at": "13:36",
        "url": "https://www.youtube.com/watch?v=xo68uCibfm8&t=816s"
      },
      {
        "text": "You don't have to worry about context engineering. You can kind of just throw the RLM at it and have it figure it out.",
        "at": "12:29",
        "url": "https://www.youtube.com/watch?v=xo68uCibfm8&t=749s"
      },
      {
        "text": "The biggest promise I see here is just imagine a world where the models are actually post-trained and actually like RLM aware. I think things will get pretty crazy pretty quick.",
        "at": "20:11",
        "url": "https://www.youtube.com/watch?v=xo68uCibfm8&t=1211s"
      }
    ],
    "words": 4268
  },
  "summary_url": "/#xo68uCibfm8",
  "transcript": {
    "html": "/transcripts/xo68uCibfm8.html",
    "txt": "/transcripts/xo68uCibfm8.txt",
    "vtt": "/transcripts/xo68uCibfm8.vtt"
  }
}