{
  "video": {
    "id": "yqF6XhzbWBk",
    "title": "Inside 847 Production Clinical AI Notes — Sebastian Fox, Composo",
    "duration": 1188,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer"
  },
  "analysis": {
    "video_id": "yqF6XhzbWBk",
    "title": "Inside 847 Production Clinical AI Notes — Sebastian Fox, Composo",
    "one_liner": "A doctor-turned-eval-engineer shows that ~1 in 20 AI-written clinical notes in production carry a seriously harmful error — mostly quiet omissions and intent changes that read perfectly fine — and argues the fix isn't a better rubric but a loop that discovers failure modes from real outputs, captures expert judgement, and retrieves the relevant judged cases into the judge's context per output.",
    "summary": "Seb Fox (medical doctor, now at Composo) opens with a note that reads like a routine tension headache but omits the jaw pain on chewing that makes it giant cell arteritis — a sight-threatening same-day emergency. He argues the dangerous failures in high-stakes AI are the ones that look completely fine: additions, changes and omissions that are locally faithful but wrong about what mattered. Judging which difference matters is \"taste\" — tacit, contextual and moving — so a pre-specified rubric only encodes the taste you could write down; his own best-practice judge waved through notes where 1 in 5 clean passes still hid a serious error. His proposal is a repeating discover → capture → calibrate loop, keeping taste as retrievable examples (past judgements, expert corrections, guidelines) assembled per output rather than in a frozen rubric or in fine-tuned weights.",
    "key_points": [
      "Opening case: an AI note records a new headache in a woman over 50 as likely tension type; the omitted line — jaw ache on chewing — makes it giant cell arteritis, which untreated can take her sight within days. \"Nothing in the note is technically wrong.\"",
      "Blatant case: a man in his 20s seen for tonsillitis gets chest pain, suspected angina, diabetes medications he's never taken and a non-existent hospital address written into his record — weeks later he's invited to diabetic eye screening for diabetes he doesn't have.",
      "Largest real-world study of these notes: ~1 in 20 carried an error serious enough to cause significant harm; widening the lens, nearly 1 in 5 had an important omission and more than 1 in 10 a hallucination — in production, on real patients.",
      "Ambient scribes are already in about a third of US practices and climbing, physician AI use doubled last year, and there is no adverse event reporting — \"it's not that we checked and it's fine, it's that we're flying blind.\"",
      "He generated notes across three of the best production ambient scribes in an afternoon and plotted every failure by how much it matters vs whether a strong automated check catches it — almost everything sits below the catch line, including the high-stakes misses.",
      "Concrete subtle failures: \"it just happened\" recorded as \"abrupt sudden onset\" (a red flag for a brain bleed the patient never said); a plan the doctor and patient explicitly talked out of (\"arrange tests today\") kept instead of the one they chose — faithful to the words, a lie about the intent.",
      "Transcription-layer errors are real and hard (Humalog heard as Humulin, hyperthyroidism → hypothyroidism, \"no evidence of cancer\" → \"evidence of cancer\") but most of what goes wrong happens with a perfect transcript: the model adds, changes or omits, and the hard part is telling whether it matters.",
      "The verification-asymmetry argument breaks here: in maths and code the verifier is free (unit test, compiler), but for \"is this note safe and complete\" you build the verifier yourself, and verification is only easier for the easy bit — spot the difference. Deciding which differences matter is harder than writing a plausibly good note.",
      "He built the best judge he's seen in teams — transcript + note + context, detailed faithfulness rubric with worked pass/fail examples, rubric auto-optimised, deterministic NLP counting differing medical concepts — and 1 in 5 of its clean passes still hid a serious error, usually an omission.",
      "Taste is tacit (experts can't write it down), contextual (the same detail is critical in one note, noise in the next) and moving (models, guidelines, hospital definitions, two good doctors disagreeing). Illustrated by two haematuria notes both dropping a holiday line: France is irrelevant, Lake Malawi means schistosomiasis until proven otherwise.",
      "Three places to keep learned taste: the prompt/rubric (fails), the weights via fine-tuning or continual learning (goes stale, can't explain why, needs a retrain), or the examples themselves — retrieved per output, live on the next call, and you can point at exactly what moved the score. \"For this problem, it's both better and also cheaper.\"",
      "Benchmark of three judges on the same generated-note dataset: a strong off-the-shelf frontier-model judge with a rubric is \"better than a coin flip\" but misses most of what matters; the serious rubric-plus-deterministic-checks system does better but still misses a lot; the loop-driven judge performs a lot better — \"the only thing that changes is what the judge was shown.\""
    ],
    "takeaways": [
      "Start with your experts leaving free-form comments on real outputs — his single \"if you take one thing away.\" A focused few hours of clinician comments, not a month-long labelling project; capture the reasoning and corrections, not just a score.",
      "Discover your failure mode ontology by clustering real production outputs, not by guessing on a whiteboard. Synthetic test cases only cover the failures you already imagined, and the ways a real system goes wrong are effectively unbounded.",
      "Stop trying to pre-specify the standard in one rubric. Write down only the generic part (\"be faithful\", \"don't drop anything important\") and assemble the case-specific standard on the fly — retrieve the most similar previously judged outputs and how they scored, the expert corrections that apply, and the reference guidelines, into the judge's context per output.",
      "Don't assume adding an LLM judge is a safety net: a judge that can't tell what counts is \"a second silent failure that just nods along with the first.\" Measure your judge against expert-labelled real outputs before trusting its clean passes.",
      "Prefer retrieved examples over fine-tuned weights when the standard is still moving and the score must be explainable — you can add a case and it's live on the next call, and you can point at what moved the score.",
      "Map it to your own domain: contract review that misses the clauses that change the deal, a support agent promising a refund you don't offer — anywhere being confidently wrong has a cost is watched, if at all, by a judge with no taste for what matters."
    ],
    "topics": [
      "evals",
      "llm-as-judge",
      "healthcare-ai",
      "ambient-scribes",
      "hallucinations",
      "context-engineering",
      "human-in-the-loop",
      "high-stakes-ai"
    ],
    "tools": [
      "Composo",
      "ambient AI scribes (three unnamed leading production products)",
      "GPA (named as the rubric auto-optimiser)",
      "RLHF",
      "deterministic NLP medical-concept matching",
      "frontier LLM judges"
    ],
    "quotes": [
      {
        "text": "The dangerous failures are often the ones that actually look completely fine.",
        "at": "01:01",
        "url": "https://www.youtube.com/watch?v=yqF6XhzbWBk&t=61s"
      },
      {
        "text": "So the note passes confidently and you put a judge like that in front of your system, you've not added a safety net, you've added a second silent failure that just nods along with the first.",
        "at": "10:48",
        "url": "https://www.youtube.com/watch?v=yqF6XhzbWBk&t=648s"
      },
      {
        "text": "A rubric that you pre-specify is only the taste you could write down. The taste that matters is the part that you couldn't.",
        "at": "11:36",
        "url": "https://www.youtube.com/watch?v=yqF6XhzbWBk&t=696s"
      },
      {
        "text": "Evaluation isn't something you have, it's something that you do continuously over time.",
        "at": "19:22",
        "url": "https://www.youtube.com/watch?v=yqF6XhzbWBk&t=1162s"
      }
    ],
    "words": 3821
  },
  "summary_url": "/#yqF6XhzbWBk",
  "transcript": {
    "html": "/transcripts/yqF6XhzbWBk.html",
    "txt": "/transcripts/yqF6XhzbWBk.txt",
    "vtt": "/transcripts/yqF6XhzbWBk.vtt"
  }
}