{
  "video": {
    "id": "Fa3hQVIKShw",
    "title": "Just-in-Time Ontologies: What Cheap Intelligence Means for Data — Hrishi Olickel, Southbridge AI",
    "duration": 1263,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": ""
  },
  "analysis": {
    "video_id": "Fa3hQVIKShw",
    "title": "Just-in-Time Ontologies: What Cheap Intelligence Means for Data — Hrishi Olickel, Southbridge AI",
    "one_liner": "There is no universal schema: meaning depends on the decision being made, so build ontologies 'just in time', at the moment of decision. Cheap decision models, together with self-optimizing 'Pareto serving' pipelines, made an Ohio campaign-finance entity-resolution job projected at about $10,000 cost $795.",
    "summary": "Olickel argues that today's data infrastructure assumes transformation is expensive. He says cheap on-tap judgment breaks that assumption. Single ontologies fail because four schemas only meet at the moment a decision is made: source, storage, decision, and the model's own prior. He separates three roles for models in pipelines: designers, workers and hill-climbing 'servers'. His case study is a 10-day, mostly agent-driven entity-resolution run over Ohio campaign finance using the decision model 'Jev'. He ends with the four things models still need to self-optimize, and a six-stage process for 'Pareto serving' a workload on cost, speed and accuracy.",
    "key_points": [
      "There are four schemas: source (what you had access to), storage (what you can record), decision (why the data is needed) and the model's prior. You only know all four when the decision is made. Example: 'customer' means whoever pays to billing, whoever calls to support, and whoever we're on the hook for to compliance.",
      "Data work falls into three regimes: write once, run once; write many, run once (e.g. large migrations); and write many, run many (repeatable pipelines, the most common).",
      "Models play three roles that coding agents mash into one context window. Pipeline designers want slow, 'thick-token' models (Opus or up). In-pipeline workers want cheap, fast, specialized models that follow instructions. 'Hill climbers' or servers self-optimize toward the optimal solution. Decision models like Jev fit the worker slot.",
      "Ohio entity resolution, development set of 200 held-out families: a cheap reasoning model alone matched Fable at about 2% of the cost. Jev missed 9 household links by confidently saying no. Rewriting the question to define 'enough evidence', without touching models or thresholds, flipped Jev from no to yes on all 13 development questions.",
      "Statewide, over a dozen qualification rounds each fixed one definition (committee, filer, two people at one address). All 203,317 cases got a label saying what the answer rests on and how far it was checked. Total cost was $795 against a preset $800 budget, versus about $10,000 expected before Jev. Checking cost about four times as much as doing: $545 on second opinions, disagreement settling and audit, against $128 for first answers.",
      "Their agent, running on Astra and later Opus, found most approaches on its own over 10 days. It tried over a dozen architectures, ran Hanks more than 4,000 times alongside 90,000 calls to Jev, and used about 6 billion tokens. Each iteration was a version-controlled Hank that was easy to revert and critique.",
      "Model priors are 'quickly becoming the largest class of error'. Examples: looking for 'in_kind' when the file used 'In-Kind', which counted 15,658 in-kind contributions as ordinary money; trying four date formats when the file used a fifth, leaving 99.4% of dates empty; storing absolute values of negative loans, which turned a $784K net into $5.3M of debt that multiple models validated.",
      "Cross-model prior intersection works: Jev and Luna agreed on 194 of 200 families, none of the shared answers were wrong, and every error was in the 6 disagreements. Models from different families disagree more. The exception is models distilled from another company's models. Some errors are shared by LLMs 'as a species': four model families produced the same total because they shared a sign convention the answer key didn't."
    ],
    "takeaways": [
      "Stop chasing a single universal schema. Merge source context with decision context when the decision is known, and keep each ontology as a reusable intermediate 'with its receipts'.",
      "Use different models for designing pipelines (slow, heavy reasoning) and working inside them (cheap, fast, specialized). Add a separate optimizing layer that swaps in cheaper or faster parts against a baseline.",
      "Define a failure surface before you start: a manually checked baseline from good models on a small slice, schema checks, or behavioural checks. Bound every experiment on cost, time and access.",
      "Budget for verification: use cross-family model disagreement to find errors, expect checking to cost more than doing, and fix definitions in the prompt before touching models or thresholds.",
      "Follow the six stages: inspect and split the job into typed questions; set the failure surface; run minimal bounded pilots; move working pieces into reusable rigs and 'codons'; deploy; let it mature in production while you Pareto-surf toward cheaper, faster options."
    ],
    "topics": [
      "ontologies",
      "data pipelines",
      "entity resolution",
      "model selection",
      "pareto optimization",
      "evals",
      "self-optimizing agents",
      "cost"
    ],
    "tools": [
      "Southbridge AI",
      "Hank Weave (as transcribed; their open-source harness for data work)",
      "Jev / Jeff (decision model, name as transcribed)",
      "Fable",
      "Opus",
      "Astra",
      "Luna",
      "GPT-3.5 Turbo",
      "Claude Code ('cloud code' in transcript)",
      "Pi ('pie' in transcript)",
      "Duct Tape B (Hannes's article, as transcribed)"
    ],
    "quotes": [
      {
        "text": "There is no concept of a customer that's divorced from the question that you want to answer.",
        "at": "04:41",
        "url": "https://www.youtube.com/watch?v=Fa3hQVIKShw&t=281s"
      },
      {
        "text": "And if you know statistics, you can explain really well why AB testing 50 users is a bad idea. Not because you're not going to get a result, but because you will.",
        "at": "07:51",
        "url": "https://www.youtube.com/watch?v=Fa3hQVIKShw&t=471s"
      },
      {
        "text": "because the moment a number exists, it argues for a result.",
        "at": "08:04",
        "url": "https://www.youtube.com/watch?v=Fa3hQVIKShw&t=484s"
      },
      {
        "text": "Checking cost about four times as much as doing",
        "at": "12:31",
        "url": "https://www.youtube.com/watch?v=Fa3hQVIKShw&t=751s"
      }
    ],
    "words": 4157
  },
  "summary_url": "/#Fa3hQVIKShw",
  "transcript": {
    "html": "/transcripts/Fa3hQVIKShw.html",
    "txt": "/transcripts/Fa3hQVIKShw.txt",
    "vtt": "/transcripts/Fa3hQVIKShw.vtt"
  }
}