{
  "video": {
    "id": "qflLT3SoVbw",
    "title": "Stop Fine-Tuning to Fix Retrieval Problems — Anant Srivastava",
    "duration": 1212,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": ""
  },
  "analysis": {
    "video_id": "qflLT3SoVbw",
    "title": "Stop Fine-Tuning to Fix Retrieval Problems — Anant Srivastava",
    "one_liner": "Prompt, memory and weights are three tools for three different jobs, not a ladder to climb: the prompt holds behaviour, memory holds facts that are current, large, citable or access-scoped, and weights hold only stable reflexes. So if the model gives wrong answers, fix retrieval instead of fine-tuning.",
    "summary": "Srivastava argues that most enterprise teams decide where knowledge lives by accident. They escalate from prompt edits to retrieval to fine-tuning, and every prompt edit, indexed document and training sample is an unplanned architecture decision. The result is an architecture that 'was accumulated, not designed'. He gives a diagnostic for each of the three layers, chosen by what the knowledge is and how fast it changes. He also describes a 'circulating' harness in which context feeds memory, memory feeds fine-tuning, and fine-tuning changes what is worth retrieving.",
    "key_points": [
      "The failure is an 'accident' in two ways. Teams escalate: they fix the prompt, then retrieval, then fine-tune. And every routine prompt edit, indexed document or training sample silently decides what lives where, with nobody owning the whole.",
      "Cautionary example: a support assistant gets prompt tone fixes from a PM, refund-policy docs added to retrieval, and the product catalog shoved into the prompt. Then it is fine-tuned on six months of support tickets. After a new catalog launches, it still invents product names that don't exist, because the old catalog leaked into the weights through the tickets.",
      "Prompt (which also covers authored instructions such as MD files, not just the system prompt) is for behaviour, tone and persona: knowledge that is small, stable and editable. Example: a SaaS support agent that keeps a professional tone, offers human escalation after three failures and gives concrete next steps. Storing facts like a product catalog in the prompt wastes tokens and causes the lost-in-the-middle problem.",
      "Memory covers both agent memory and external enterprise memory retrieved via RAG. It is for knowledge that is current (changes faster than you can fine-tune), large and citable, and for anything scoped per user or with access control. It is not for behaviour or reasoning: if a model can't reason over 2, 3 or 5 documents, giving it 50 won't help.",
      "For a code assistant over an organisation's repos: use code-aware chunking, and denormalise metadata (which repo, which users may commit) onto each chunk so filtering returns exact data instead of a 'RAG mush' that confuses the model.",
      "Weights are for what has stopped changing, and 'the only axis' is rate of change. Anti-pattern: fine-tuning an internal docs assistant on runbooks and procedures to 'teach the domain'. That was really a retrieval problem, and the result is stale answers.",
      "A valid fine-tuning case is a hard, ambiguous task such as content moderation or claims processing. Start with a frontier model and inconsistent human corrections. Once override rates flatline, a stable centre emerges alongside a contested edge that still needs humans. Fine-tune on the centre and monitor it for drift. For medical ICD-10 coding (about 70,000 codes; Mount Sinai and IMO Health are cited), don't fine-tune the codes themselves, which are facts. Fine-tune the reflex of understanding input formats and choosing between them.",
      "Fine-tune for one of two reasons: capability or cost. With frontier models it is usually cost, meaning a small fine-tuned model running high volume cheaply on a stable pattern.",
      "The target architecture is a loop. Context window signals become durable memory, and memory is pulled into context at session start. Over time, retrieval patterns and formats move from memory into weights, and once fine-tuned you no longer need to retrieve format examples. In his words, the agent 'gets better at its job by the virtue of doing its job.'"
    ],
    "takeaways": [
      "Before any change, ask where it should live: Is it small, stable and about how to behave? Prompt. Is it too large to fit, changing faster than you can retrain, or scoped per user, repo or permission? Memory. Has it stopped changing, have the humans converged, and are you fine-tuning for capability or cost? Weights.",
      "When the model gives wrong answers about documentation, runbooks or procedures, fix retrieval (the right chunks) rather than fine-tuning. Fine-tuning freezes facts that will go stale.",
      "Keep facts like product catalogs out of the prompt and out of fine-tuning data. Check training data such as old support tickets for facts that could leak into the weights.",
      "In RAG, use structure-aware chunking (code-aware for code) and attach denormalised metadata (repo, permitted users) so you can filter precisely and enforce access control.",
      "Build a harness that moves knowledge between context, memory and weights deliberately. Fine-tune only on a stable centre where human overrides have flatlined, and monitor that centre for drift."
    ],
    "topics": [
      "rag",
      "fine-tuning",
      "prompting",
      "agent memory",
      "context engineering",
      "llm architecture",
      "enterprise ai"
    ],
    "tools": [
      "Mount Sinai",
      "IMO Health"
    ],
    "quotes": [
      {
        "text": "your architecture was accumulated. It was not designed. That is the accident.",
        "at": "04:36",
        "url": "https://www.youtube.com/watch?v=qflLT3SoVbw&t=276s"
      },
      {
        "text": "if your model cannot reason over two or three or five documents, it won't be able to reason over 50 that you provide.",
        "at": "08:32",
        "url": "https://www.youtube.com/watch?v=qflLT3SoVbw&t=512s"
      },
      {
        "text": "So if your if your model is not giving the right answers, fix the retrieval problem. Don't just go ahead and fine-tune your model.",
        "at": "13:46",
        "url": "https://www.youtube.com/watch?v=qflLT3SoVbw&t=826s"
      },
      {
        "text": "the agent gets better at its job by the virtue of doing its job.",
        "at": "19:18",
        "url": "https://www.youtube.com/watch?v=qflLT3SoVbw&t=1158s"
      }
    ],
    "words": 3261
  },
  "summary_url": "/#qflLT3SoVbw",
  "transcript": {
    "html": "/transcripts/qflLT3SoVbw.html",
    "txt": "/transcripts/qflLT3SoVbw.txt",
    "vtt": "/transcripts/qflLT3SoVbw.vtt"
  }
}