{
  "video": {
    "id": "WP3hjUXd918",
    "title": "Context Engineering in 2026 — Louis-François Bouchard, Omar Solano & Samridhi Vaid, Towards AI",
    "duration": 3806,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": "In person"
  },
  "analysis": {
    "video_id": "WP3hjUXd918",
    "title": "Context Engineering in 2026 — Louis-François Bouchard, Omar Solano & Samridhi Vaid, Towards AI",
    "one_liner": "Towards AI ran ~$600 of context-engineering experiments on their open-source AI tutor and found that with prompt caching on, not compacting at all — keeping the full history — beat every compaction preset on recall, cost and latency simultaneously.",
    "summary": "Louis-François Bouchard, Omar Solano and Samridhi Vaid walk through the compaction, offloading and retrieval techniques available in 2026, then measure them on their production AI tutor using real student questions rather than synthetic evals. The punchline is that prompt caching has changed the economics: because cached tokens can be up to 50x cheaper (DeepSeek), any summarisation or compaction invalidates the cache and forces full-price recomputation, so compaction only pays if it compresses more than ~50x without losing quality. Across Gemini 3.5 Flash and DeepSeek V4 Flash, 'full history' won on memory recall (95% vs 32% when summarising), on cost (97% of tokens cached) and on latency (clearing tool outputs makes the agent re-retrieve what it already had). Compaction only becomes necessary when a hard constraint — like a 32K local model on a MacBook — forces it, at which point retrieval, and specifically hybrid search, is the answer.",
    "key_points": [
      "Prompt caching inverts the compaction calculus: cached tokens are typically ~90% cheaper and up to 50x cheaper on APIs like DeepSeek, so compaction must compress by more than 50x to be worth the cache invalidation — 'summarization is potentially a trap'.",
      "In their eval, the 'full history' preset (touch nothing) beat both their shipped production defaults and six compaction techniques (sliding window, prompt compression, selective retention, etc.) on all three of recall, cost and latency; the production preset degraded to 38% on multi-turn, costing ~$0.50/turn single-turn and ~$0.24/turn multi-turn.",
      "Reason full history is also faster and cheaper: consistently clearing tool outputs makes the agent re-retrieve information it already had, so it issues more tool calls and burns more tokens.",
      "On DeepSeek V4 Flash, keeping everything recalled the exact detail 95% of the time vs 32% when summarising/compacting first; a 36-turn, ~1.78M-token conversation ran with 97% of tokens cached, making the setup sending the most tokens the cheapest to run.",
      "No context rot up to 800k tokens for distinctive facts; performance on ambiguous facts dropped to about half. Their sessions dataset was only 11–13 turns, which they flag as a limit.",
      "Architecture: a single LangChain React agent (create_agent + InMemorySaver + middleware), FastAPI wrapper, Next.js UI, over an 8M+ token corpus of their lessons plus LangChain/LlamaIndex/OpenAI/Claude Code docs. Hybrid search = Cohere embeddings + BM25 → top 30 → rerank to top 5, capped at 100k tokens.",
      "The Karpathy-style 'browse the knowledge base with bash' tool (raw/ + generated titles/ + a Claude-Code-generated wiki/, read-only, 8s command timeout, 40k-char output cap, 20 commands/turn) was used in ~90% of turns but produced identical recall and was 50% slower — 'it was fun to add but we didn't see any benefit'.",
      "Retrieval at long context: dense/semantic RAG held ~80% from 50k–200k tokens but hit 0% recall at 400k for facts buried in the middle, while BM25 got 100% every time — hence hybrid. Graph RAG tied with plain RAG at 100% on their real-user evals and cost far more to set up.",
      "Local (MacBook, 32K max window) breaks the strategy: lessons alone exceed the window, caching stops helping, chat recall falls to 33% vs 92–95% on cloud, and over-stuffing took ~340 seconds to emit a single token. Local RAG over pasted documents, by contrast, hit 100% accuracy in 25–65 seconds. Going from 7B → 8B → 32B does not buy you context window."
    ],
    "takeaways": [
      "Don't compact by default — name the constraint first. Measure your own presets: their 'unproven defaults' (clear tool outputs after 5k tokens keeping last 5, summarise after 30k keeping last 20 messages) were worse than doing nothing.",
      "Optimise for cache hits before you optimise for token count. Refresh the session on a genuine topic change, clear stale tool outputs, use progressive disclosure — but stop rewriting context the provider has already cached.",
      "Log everything per turn — input/output tokens, how many were cached, cost, time to first token, tool call count, and whether compaction fired — and track cache hit rate and user frustration signals. 'Just ask Claude to implement Opik.'",
      "Evaluate on real user questions, not synthetic ones (they scraped 60 cleaned Q&A pairs from their own student forum with Codex), and add gate checks that verify the technique under test actually triggered before you trust a run.",
      "Use hybrid search rather than semantic alone — dense retrieval collapses on middle-buried facts at 400k tokens where BM25 does not — and grade with an LLM via your Claude/Codex subscription rather than the API, which is cheaper.",
      "Prefer many small, cross-referencing skills over few large ones, loading them one at a time (progressive disclosure); use delta summarization when you do spawn sub-agents, though a single agent was enough for their tutor."
    ],
    "topics": [
      "context-engineering",
      "compaction",
      "prompt-caching",
      "rag",
      "evals",
      "agents",
      "long-context",
      "local-models"
    ],
    "tools": [
      "LangChain",
      "LlamaIndex",
      "Gemini 3.5 Flash",
      "DeepSeek V4 Flash",
      "Claude Code",
      "Codex",
      "Cohere",
      "BM25",
      "FastAPI",
      "Next.js",
      "Hugging Face Spaces",
      "Opik",
      "OpenAI",
      "Towards AI Academy"
    ],
    "quotes": [
      {
        "text": "summarization is potentially a trap",
        "at": "18:10",
        "url": "https://www.youtube.com/watch?v=WP3hjUXd918&t=1090s"
      },
      {
        "text": "keeping everything wins on on all of these three fronts",
        "at": "45:53",
        "url": "https://www.youtube.com/watch?v=WP3hjUXd918&t=2753s"
      },
      {
        "text": "on Deep Seek we saw um the setup that was sending the most tokens is actually the cheapest to run",
        "at": "52:23",
        "url": "https://www.youtube.com/watch?v=WP3hjUXd918&t=3143s"
      },
      {
        "text": "do not compact by default. You have to name the constraint that you have and then you know look for a better alternative.",
        "at": "61:40",
        "url": "https://www.youtube.com/watch?v=WP3hjUXd918&t=3700s"
      }
    ],
    "words": 11878
  },
  "summary_url": "/#WP3hjUXd918",
  "transcript": {
    "html": "/transcripts/WP3hjUXd918.html",
    "txt": "/transcripts/WP3hjUXd918.txt",
    "vtt": "/transcripts/WP3hjUXd918.vtt"
  }
}