{
  "video": {
    "id": "RdQCY_-SXbA",
    "title": "AI Can't Make Up a Number It Can't Look Up - Vinoo Ganesh, Kepler",
    "duration": 1108,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": ""
  },
  "analysis": {
    "video_id": "RdQCY_-SXbA",
    "title": "AI Can't Make Up a Number It Can't Look Up - Vinoo Ganesh, Kepler",
    "one_liner": "Kepler stops numerical hallucination by design: code extracts every fact into a deterministic store with hashed pointer IDs, and the model only plans and cites those pointers, so if a pointer can't be dereferenced the number can't appear in the output. Verification added only at the end misses most of this.",
    "summary": "Ganesh, CEO of Kepler and former head of business engineering at Citadel, argues that Wall Street still doesn't trust AI because of a structural problem, not just model quality. He says waiting for models to 'eliminate hallucination' makes no sense, since in finance hallucination's other name is alpha. He names three sources of wrong numbers: the wrong document, the wrong number from the right document, and model 'mental math'. Kepler's architecture tackles all three: code writes every number during preprocessing, the model acts as a planner, retrieval is wide and done by code with security scoping, and a post-processing verification step comes last. He adds that the competitor that copied Kepler's pitch copied only that last verification step.",
    "key_points": [
      "About $270M of VC money went into formal verification layers in the three months after his AIE World's Fair talk. A VC gave Kepler's pitch deck to a competitor, which copied the tagline but only the post-hoc verification piece, not the structural part that comes before it.",
      "Wrong numbers come from three sources: (1) retrieving the wrong document, (2) pulling the wrong number from the right source, e.g. reading Palantir's 2023 revenue from the 2025 10-K and confusing the years, and (3) mental math. 'Models don't do math. They predict the next token.' A single CPU cycle can add two numbers deterministically.",
      "Kepler's architecture: in a large preprocessing and contextualization step, code assigns pointers (unique IDs, shown as hashes) to facts such as a number in a 10-K, turning unstructured data into a structured store. The model works like a smart analyst that plans; code and math do the retrieval. The model can't generate a number unless it dereferences a pointer, and looking up a missing key in a hash map returns nothing.",
      "Citations end up verified, corrected at search time, or refused outright. Users can click chips on each number to see exactly how it was derived.",
      "Facts that can't be stored deterministically go into hybrid storage (Turbopuffer plus other mechanisms). Retrieval runs ANN plus BM25 for up to about 200 documents, reciprocal rank fusion down to about 60, then cross-encoders over query and chunk pairs, combined with plain pointer dereferencing ('dot get methods'). The system searches wide and then narrows, because LLMs prefer cheap depth-first search and run with their first hit. 'A lookup cannot get it wrong. Search can.'",
      "Five security failure modes in document retrieval: forgetting to filter by tenant (Citadel searching Millennium's documents), a caller impersonating another firm, stale search results, direct reads that skip search rules or allow lists, and hitting an unpopulated index. The fix is scoping every search with security built in, including which OAuth token and user scope are used.",
      "In Kepler's production data, refusals (the model saying it couldn't dereference a pointer, so it couldn't produce a number) dropped sharply but not to zero, because models are still probabilistic. That is why another post-processing step follows.",
      "He says most finance AI tools today are used to format PowerPoint presentations, and nobody has built a DCF 'cradle to grave'. Stat cited: 80% of finance execs trust the numbers, but 31% find errors in them."
    ],
    "takeaways": [
      "Build verification into the architecture rather than tacking it on: extract facts in preprocessing, give them pointer IDs (hashes), and let the model reference facts only through pointers it must dereference.",
      "Use the model to plan, and do arithmetic and retrieval with deterministic code and tools, not next-token prediction.",
      "Scope every retrieval with security: tenant filtering, caller identity checks, freshness, allow lists on direct reads, and handling of empty indexes.",
      "Search wide in code (ANN + BM25 → reciprocal rank fusion → cross-encoder re-ranking) instead of letting an LLM settle on its first hit.",
      "Questions to put to any vendor or tool: Does the model write pointers? Is every citation source-checked? Is the arithmetic source-checked? Can files be stopped from reaching clients they shouldn't? What happens to a number that came from nowhere?"
    ],
    "topics": [
      "verification",
      "hallucination",
      "rag",
      "finance",
      "retrieval",
      "security",
      "deterministic-tools",
      "citations"
    ],
    "tools": [
      "Kepler",
      "Anthropic",
      "OpenAI",
      "Turbopuffer",
      "BM25",
      "ANN",
      "reciprocal rank fusion",
      "cross-encoders",
      "OAuth",
      "Citadel",
      "Candlestick Capital",
      "Millennium",
      "Jane Street",
      "GLG",
      "Guidepoint",
      "Palantir",
      "Google Search"
    ],
    "quotes": [
      {
        "text": "You know another word for hallucination? Alpha.",
        "at": "02:16",
        "url": "https://www.youtube.com/watch?v=RdQCY_-SXbA&t=136s"
      },
      {
        "text": "Three is the mental math. Models don't do math. They predict the next token.",
        "at": "05:25",
        "url": "https://www.youtube.com/watch?v=RdQCY_-SXbA&t=325s"
      },
      {
        "text": "If a model cannot deresolve that pointer, it cannot print that number out.",
        "at": "14:11",
        "url": "https://www.youtube.com/watch?v=RdQCY_-SXbA&t=851s"
      },
      {
        "text": "We're now out of the search phase of AI.",
        "at": "17:56",
        "url": "https://www.youtube.com/watch?v=RdQCY_-SXbA&t=1076s"
      }
    ],
    "words": 3794
  },
  "summary_url": "/#RdQCY_-SXbA",
  "transcript": {
    "html": "/transcripts/RdQCY_-SXbA.html",
    "txt": "/transcripts/RdQCY_-SXbA.txt",
    "vtt": "/transcripts/RdQCY_-SXbA.vtt"
  }
}