{ "video": { "id": "Tt2kX2sgQio", "title": "How Kepler Built Verifiable AI for Financial Services — Vinoo Ganesh", "duration": 1350, "upload_date": null, "channel": "AI Engineer", "source": "AI Engineer", "event_format": "In person" }, "analysis": { "video_id": "Tt2kX2sgQio", "title": "How Kepler Built Verifiable AI for Financial Services — Vinoo Ganesh", "one_liner": "Kepler's CEO argues that citations are only half of trustworthy AI — you cannot eval a probability machine into determinism, so the model must decide *what* to compute while a deterministic substrate (atomic provenance, scope determinism, derivation chains) actually extracts and computes every number.", "summary": "Vinoo Ganesh (CEO/co-founder of Kepler, ex-Palantir compute platform lead, ex-head of business engineering at Citadel) contends that AI turned a writing problem into a reading problem: producing content is nearly free, but finance has no verification layer — unlike CI/CD and code review in software, pharmacists checking prescriptions, or pilot/co-pilot — beyond an overworked VP. He distinguishes citation (an after-the-fact audit showing where information came from) from verification (a deterministic, repeatable proof that an extracted number is correct), and notes verification is not ground truth but conformance to a firm's own rules — two Citadel TMT desks can share data and take opposite sides. Kepler's answer is three tenets: atomic provenance (the model writes a *reference* to a number, never the number), scope determinism (the model decides what to compute, code does the computing), and derivation chains (replayable, rewindable records of how a ratio was built). He argues this generalizes past finance to legal citations and drug discovery, and that the industry is roughly 'pre-SSL' — the era of token-maxing is about to give way to a cost-optimization rush like the Snowflake/Databricks ROI wave.", "key_points": [ "Evals are not verification: 'You cannot take a non-deterministic LLM and eval your way to something deterministic.' Citations get you ~50% of the way; the citation is effectively an after-the-fact audit.", "Kepler trained its own extraction model that outperformed foundation models at 94% accuracy — and he dismisses it on stage: 'Who here would trade off of something that's 94% accurate?' A wrong number is still wrong if you're in the unfortunate 6%.", "Atomic provenance: the model emits a reference to a number and hands off to a database/deterministic system to write it; it never writes or manipulates the number and 'doesn't even understand what that number is.' A deterministic check then strips out anything that can't be independently verified — explicitly NOT models cross-checking models ('have OpenAI check Anthropic and have Anthropic check XAI').", "Scope determinism: 'the model decides what to compute. It never does the computation itself.' Why run 1 + 1 through a multi-billion parameter model instead of one CPU cycle? Kepler splits the deterministic pieces of the model ('which are none') from the non-deterministic ('which are all of them') and gives it tooling — e.g. it knows it needs net margin and where to look, but code parses the PDF/XBRL and persists the value outside the model's reach.", "Derivation chains handle ratios that don't appear in filings — gross margin, enterprise value, EBITDA adjustments — because every firm calculates multiples and recurring/non-recurring differently; the chain of events is replayable and rewindable, mirroring what an analyst does on a hedge fund desk.", "Verification is not ground truth: it's verifying the output respects 'the nouns and verbs or the rules of your organization.' Two TMT desks at Citadel with identical data and identical verification mechanisms can be long and short the same stock.", "He claims people buy Bloomberg and FactSet 'to displace culpability' — the data is free in SEC filings, but if it's wrong, everyone on Wall Street has the same incorrect information. And every such tool is read-only, which is the wall blocking real adoption: analysts still consume and hand-produce the work product.", "Result: consolidated financial statements in seconds with every number tied back to its source, and a DCF built in the format you want without hallucinated rows — displacing the contractor-driven data operations at CapIQ-style vendors.", "The pattern generalizes: pre-processing that deterministically extracts entities (case A v. case B) so a Harvey or Legora never hallucinates a citation, or NIH white-paper compounds that are never missed.", "Token-maxing is ending: companies giving bonuses for consuming a billion tokens ('gold cards' at this very conference) is the only time in history 'an employee been rewarded for your company to pay another vendor.' He predicts a cost-optimization rush mirroring the Snowflake/Databricks ROI wave he spoke about on this stage 4 years ago." ], "takeaways": [ "Stop trying to fine-tune or eval your way to reliability on numerical tasks — 94% is a demo, not a tradeable product. Move the number-handling out of the model entirely instead of improving the model's accuracy.", "Architect so the model plans and selects while deterministic code executes: let it decide it needs net margin and which filing to hit, but have real code parse the XBRL/PDF, compute the arithmetic, and persist the value where the model can't touch it. It's also cheaper.", "Have the model emit references, not values, and run a deterministic check that strips any number you can't independently re-verify — don't build a panel of LLMs grading each other's work.", "Record derivation chains, not just citations, so a ratio can be replayed and rewound — and encode your organization's own definitions (what counts as recurring, how you compute enterprise value), because verification means conforming to your firm's rules, not to a universal ground truth.", "Build for the AI analyst, not the AI portfolio manager. Customers are most excited about giving back time from repeatable pain — listening to earnings calls, opening 15 tabs of 8-Ks and 10-Ks to build the V0 of a model — done by people earning 600–700k a year." ], "topics": [ "verifiability", "provenance", "determinism", "fintech", "evals", "agents", "rag", "cost-optimization" ], "tools": [ "Kepler", "Anthropic", "Claude", "Claude for Science", "Opus 4.8", "GLM 5.2", "Hugging Face", "ChatGPT", "OpenAI", "xAI", "Palantir", "Citadel", "Bloomberg", "FactSet", "CapIQ", "Daloopa", "Tegus", "Seeking Alpha", "Reddit", "Substack", "Harvey", "Legora", "Snowflake", "Databricks", "SEC", "OCC", "NIH" ], "quotes": [ { "text": "Evals are not verifiable. You cannot take a non-deterministic LLM and eval your way to something deterministic. These are probability machines.", "at": "02:00", "url": "https://www.youtube.com/watch?v=Tt2kX2sgQio&t=120s" }, { "text": "AI has made a writing problem a reading problem.", "at": "02:11", "url": "https://www.youtube.com/watch?v=Tt2kX2sgQio&t=131s" }, { "text": "It outperformed foundation models, it's 94% great. It's in the article. Who here would trade off of something that's 94% accurate?", "at": "10:22", "url": "https://www.youtube.com/watch?v=Tt2kX2sgQio&t=622s" }, { "text": "The model decides what to compute. It never does the computation itself.", "at": "12:42", "url": "https://www.youtube.com/watch?v=Tt2kX2sgQio&t=762s" }, { "text": "In what time in history has an employee been rewarded for your company to pay another vendor for how much money you're spending?", "at": "17:23", "url": "https://www.youtube.com/watch?v=Tt2kX2sgQio&t=1043s" } ], "words": 4489 }, "summary_url": "/#Tt2kX2sgQio", "transcript": { "html": "/transcripts/Tt2kX2sgQio.html", "txt": "/transcripts/Tt2kX2sgQio.txt", "vtt": "/transcripts/Tt2kX2sgQio.vtt" } }