AI Can't Make Up a Number It Can't Look Up - Vinoo Ganesh, Kepler

AI Engineer · 18 min · 216 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:12Hi everyone.
00:13My name is Venu Ganesh.
00:15Um, excited to be here today.
00:16Uh, I'm the CEO and co-founder of Kepler.
00:20Um, this is going to be kind of an interesting talk because it comes from what happened 3 months ago.
00:26So, 3 months ago, uh, I think there's a little bit of an overlap, but I gave a talk at AIE World Fair, uh, uh, really around verification, um, and why that
00:35most institutions need verification in some form of the trust ecosystem of AI.
00:42Well, yesterday most of you saw Jack and I were on stage talking about why Jack, who ran Citadel's biggest book, and started Candlestick Capital, still doesn't trust AI.
00:52Since that 3-month period, there's been about a 270 million dollars of VC money that got poured into formal verification layers.
01:03A VC actually took Kepler's own pitch deck and gave it to a competitor, and produced something that has our same tagline.
01:10I'm not bitter, just a little, but the funny thing is they copied the wrong part.
01:14And so, what I want to do is actually give you, the practitioners today, you can't just tack verification on at the end.
01:21And so, uh, when I gave that chat or that talk a few, um, months ago, uh, Anthropic had just published an article on how Kepler does what it does, and why it's really powerful and and really unique.
01:33So, what I want to do is get into the technical side of how we do the thing that we do, and ultimately this is driven by one very simple insight,
01:42Wall Street still doesn't trust AI.
01:45Most people think it's because of this.
01:47Uh, AI will estimate things.
01:49It will hallucinate things.
01:50A frontier model will make stuff up.
01:53And ultimately it is, and that's a component of it.
01:56But the actual core underlying problem is we have a structural problem in this technology that we're all so desperate to use.
02:03Meaning, if you actually chat with people that are building rag platforms, most of what they'll say is, "Well, the models will eventually get good or get better to the point they will eliminate hallucination."
02:14That is a nonsensical term.
02:16You know another word for hallucination?
02:18Alpha.
02:20Alpha is how a firm makes money.
02:21The creativity that you have and that your analysts have is a mechanism through which you can be a unique firm.
02:28Otherwise, there should be only one company on Earth.
02:31So, when you ask these types of simple questions around where data comes from, there is a piece of this that the model will just make things up, and there's a piece of this that your firm's internal data is never actually reflected
02:42in this setup.
02:45So, what are we seeing right now?
02:46We're seeing everyone with a checkbook buy everything they can.
02:50Every rag platform, some firms are trying to build, some firm some firms are running agent pilots.
02:56I had a a customer meeting this morning where someone said they were actually an AI-first firm, which means they have one co-pilot agent running all the stuff.
03:04So, why are we not seeing the complete switch of fundamental investing becoming effectively, why are we not all Jane Street?
03:12Why Why is that not happened yet?
03:14And so, the challenge really comes from this idea of in order for you to produce work product that you believe in, you have to trust it, meaning you have to blame somebody who gets it wrong.
03:26So, we see numbers like this all the time, where 80% of finance execs trust numbers, but then 31% find those errors.
03:33So, what do we actually need?
03:36We need a situation where the reviews that we're getting for AI apps, as builders ourselves, do more than just ragify things.
03:45And And these are quotes we've actually pulled from specific uh I don't I don't want to say our competitors, but other people, where things demo super well and they behave like a really smart intern.
03:55Or things are right if you just hook up the right structured information, but that last 10% still gets things wrong.
04:03So, what I want to do is we're lucky, we have great partners and folks like Turbo puffer and I gave a similar chat yesterday about this, but how do we actually do the verification?
04:11How do we architecturally build a system that does not hallucinate numerically?
04:17So, let's talk about it.
04:18Now, wrong numbers or wrong facts come from three sources.
04:23Either one, you retrieved the wrong document.
04:26So, I pulled a document, some random thing from somewhere else.
04:30How do you fix this?
04:31Well, by retrieving the right document.
04:33But, that in itself is complicated.
04:35Is the right document my own firm's internal analysis of what this stock is going to be worth?
04:40Is it, let's say, uh some analyst who I really trust more than anyone else?
04:45That's piece one.
04:47Now, failure two.
04:48We get the right source and we pull the wrong number out of that source.
04:52Let's say we're trying to figure out what the revenue for, I don't know, Palantir was back in 2023.
04:58Well, if I look at the 2025 10K that says Palantir's 2023 revenue was this, I better make sure that that number corresponds to 2023 and not 2025.
05:08So, the fix here is really we have a filter that actually checks each citation.
05:13But, this is still a problem because that filter only applies after the fact.
05:18And formal verification doesn't actually help here because the thing has already been produced.
05:23Three is the mental math.
05:24Models don't do math.
05:26They predict the next token.
05:28Now, for those of you who are going to say we have deterministic models, great, we do.
05:32We also have one CPU cycle that's good at adding two numbers together.
05:36Why don't I use that CPU cycle?
05:38So, third failure is let's use deterministic tools that we already have to actually compute numbers and entities rather than relying on some external tool.
05:49So, here's what we do.
05:50This is the magic behind Kepler, at least for piece one, there's a whole bunch of other stuff.
05:54We are architected fundamentally differently.
05:57Code writes every number and what code does in pre-processing is we assign pointers to unique identifiers.
06:03Meaning facts.
06:05Meaning something like a number in a 10K.
06:08This operates in a giant pre-processing and contextualization step.
06:12Such that all of this exists, going from unstructured data to a structured data store.
06:17Then, the model is effectively a smart analyst.
06:20It figures out how to go about solving the problem, but not actually being the thing to solve the problem itself.
06:27So, it plans.
06:29It says, "I'm going to go here and retrieve this source and retrieve this entity and do this thing."
06:34Code and math do the retrieval.
06:36By actually pulling a number out of a 10K at like a search time, that's a hugely expensive token cost.
06:43And Anthropic and OpenAI should thank you for paying all those tokens in token maxing, but most of your firms don't like that, at least anymore.
06:49I know there were people getting bonuses from token maxing for a while.
06:52Um Now, the next piece of this is really once the model's actually read both these pointers, it can't structurally generate numbers.
07:02Meaning unless the model actually has a mechanism of dereferencing that pointer, that number or that fact cannot be generated in Kepler's output.
07:12Now, we do run a post-processing step, which is the verification step.
07:16And this is the piece that was kind of funny where the last company just copied the verification piece, but all the structural stuff beforehand didn't make it.
07:24So, what you as a consumer actually see is you see something like the revenue is this amount to this amount, and you physically can click on these chips and see exactly how we got the information.
07:35Now, in my past life, I had an opportunity.
07:37I I was head of business engineering at Citadel.
07:39So, I worked across um the firm to figure out effectively what do we need to produce software that helps us generate alpha.
07:46You know the crazy thing with Citadel?
07:48PMs are so talented and so smart, but they work when they're paired with deterministic tools and technology.
07:55And that's kind of the magical insight behind fundamental investing.
07:58A fundamental investor so good at understanding data, but that data has to be right.
08:04And so, when you build a system like Kepler, what you effectively have to do is look up the right information and then be clever about the information you don't have.
08:14So, let me give you the parallel.
08:16Most folks and most buy-side researchers or buy-side investors try to use tooling or build DCFs on their own and then they find and search for information they don't have.
08:27They schedule GLG or Guidepoint calls.
08:29They find experts.
08:30They speak to management.
08:32The same system actually applies here.
08:34So, what we do is we pick the path and we store the information deterministically that we can actually store deterministically.
08:41When we have information we can't store deterministically, we run similar preprocessing steps to grab information, store that information in a hybrid storage mechanism, which is a combination of Turbo Buffer,
08:53which is awesome by the way, and a bunch of other um mechanisms.
08:56And then we ultimately use a combination of re-ranking with literally old-school like pointer dereferencing like dot get methods to pull things into our system.
09:07Now, why this matters is a lookup cannot get it wrong.
09:13Search can.
09:14If you're running the same 10 searches with the same query, the probabilistic nature of models means there's inherent entropy.
09:21That's actually an opportunity.
09:23And so, that's what we've changed here.
09:26Now, how we do this is we break this down and this occurs across of the SEC world as well.
09:31Um we are frequently throwing candidate documents away.
09:36Meaning we consistently have to look at not only what source is the right source for what information, but also which documents should we stop looking at kind of globally.
09:45So, uh we run ANN plus BM25 to get back a set of documents.
09:50It's usually up to 200.
09:51Uh we do reciprocal uh ranking We effectively get 60 documents back.
09:55It's probably the best way to think about this.
09:57Um then we do cross-encoders.
10:00We try to figure out exactly the query and the actual document chunked together and get those results that way.
10:08What that allows us to do is we can search really wide.
10:11You know what LLMs aren't good at doing?
10:13Searching really wide.
10:15What they try to do is create a plan and find either the fastest or easiest breadth-first search mechanism actually made depth-first search mechanism of actually like getting the first document and moving forward.
10:26So we allow this to search really wide and then narrow from there.
10:31The reason LLMs don't like searching wide is because it's expensive.
10:35It's because looking across a bunch of different sources is significantly worse than getting the reward function back with your first hit and going from there.
10:46Okay, so um I want to talk a little bit about security here cuz it's super important.
10:51Kepler was built with a security-first mindset and security-first mentality.
10:55Now client documents, and this is specifically targeted towards the builders here, this is a thing that no one's chatting about.
11:02No one is actually talking about the security of what OAuth token is used and what user-scoped mechanism of retrieving documents, which is actually pretty scary.
11:11So let's talk about how things go wrong in these systems that people are trying to build and how you can fix it.
11:16First and foremost, you forget to filter the tenant.
11:19All of a sudden Citadel can search through all the Millennium's documents, you will be in a lot of trouble.
11:23Second, a caller claims to be another firm.
11:26Meaning I can claim that I'm impersonating this other entity and querying documents that way.
11:32Third, search returns information that's out of date.
11:35We've had this problem kind of globally and even Google search results.
11:39Fourth, direct read skips the search rules.
11:41Meaning you're not actually querying the full corpus of information or like there's not an allow list, you only get back the single document that you did the search.
11:48Five, you hit an index that isn't populated.
11:52The way to actually solve all these problems, and there's individual things listed here, is actually scoping searches with security built in.
12:01That's kind of another critical piece that I think a lot of people miss when they're trying to build these systems themselves.
12:06Inevitably, people will start caring about the security associated with querying this information.
12:11Alpha is the most elusive thing out there, especially for folks on Wall Street.
12:15So, making sure you have a good mechanism of actually constraining the set of documents that come back with each query is really important.
12:25Okay.
12:26Now, let's talk about verification, which is the last piece here.
12:29This is probably what we're known most for, but we do the other stuff I mentioned as well.
12:33Um how we do this is by pre-processing information and storing it deterministically.
12:40We have systems like code that actually write individual pointers.
12:45The model can only then read those pointers and grab information back if they exist in the database.
12:50If I grab something out of a hash map that doesn't exist, I don't get anything back.
12:55And so, this combination is what allows us to actually make sure the information that we retrieved actually came from the source that it came from.
13:03Now, this is a kind of hidden version of what our source IDs look like, but these are hashes, and these are very simple things that you can do in your own system and your own setup to ensure that you're pulling back facts that in fact do exist.
13:19Cool.
13:20So, that's a little bit about how we do verification.
13:22Now, I want to chat a little bit about what this enables kind of globally.
13:28This enables us to verify facts that come out of our grounded corpus of data.
13:34Now, you as a builder, or you as a startup creator, or you as someone who's trying to figure out how do I trust my outputs, really just need you to do a couple of things.
13:43You need to show people that as you're creating these agents that are theoretically looking for alpha, or whatever they'd be looking for, you're using information that is actually relevant
13:52and concrete, and most importantly accurate, to the set of people that you're trying to service.
13:58So, we end up with a contract where citations are either verified, corrected at search time, or flat out refused.
14:08This is crazy if you think about it.
14:11If a model cannot deresolve that pointer, it cannot print that number out.
14:17So, that's kind of an insane thing.
14:19It means that the whole and not just a number, let's talk about facts.
14:23Like we see these court cases all over of precedents being hallucinated.
14:28Well, why can't we create a system that just has the precedent stored deterministically and allows us to search through that list and not have a random court case that never existed all of a sudden start existing?
14:39So, this tech and this this mechanism and this technology actually becomes fairly globally applicable.
14:44And the reason I'm on stage telling you how we do what we do is because I'm terrified that Wall Street never trusts this technology because of the fear that so many other companies or so many build-it-at-home
14:56folks have been uh instituting in these organizations.
15:01It's It's gotten to the point where the vast majority of finance AI tools right now are being used to format PowerPoint presentations.
15:08We still haven't actually built a DCF cradle to grave despite what most people are saying.
15:13And if we had, every single company on Wall Street would have adopted that overnight.
15:18So, this is my fear.
15:19My fear comes from a very simple fact.
15:21For us to actually get value out of this technology, we need to be able to build technology that people actually can trust and can verify.
15:29That isn't just true in finance, it's true in a bunch of different industries.
15:32So, please take what you need to from this presentation and get people using this technology, otherwise we're all going to be out of a job.
15:40Now, I want to try through what happened when we saw this.
15:42So, when we went through this whole process, um this is actually some of our our real production data here.
15:49Um refusals are when the model does not is not able to produce a number.
15:55Meaning, the model will say, "I couldn't dereference this pointer, therefore I could not produce this number."
16:02We saw that drop super drastically.
16:04Now, the reason that it didn't drop to zero is because models are still probabilistic.
16:10Meaning, it can still make up numbers after the fact during inference.
16:14So, there was another pre- a post-processing step after this that we we have as well.
16:18But my point with this slide is just by creating a system where the models have to dereference their own pointers, you can actually create a very powerful, very verifiable
16:29AI platform.
16:31And that to me is my ask to all of you.
16:33As you go through the building process, as you go through the scoping process, any tool that you use, ask them if you can actually verify the numbers.
16:40Ask them if you can see where the sources are, not in terms of some after-the-fact annotation, but click on the number, whether it's a calculation, whether it's an extraction,
16:50or some combination of the two, to make sure you trust the insight.
16:54And all of this ultimately is going to be the unlock.
16:58So, you as a vendor or you as a builder, ask these questions.
17:03Does the model actually write pointers?
17:04Does it actually create a mechanism for verifying the output?
17:07Is every citation source checked?
17:10Is arithmetic source checked?
17:12Do you have a mechanism for stopping files that reach clients or customers that it shouldn't reach?
17:18And most importantly, what happens to a number that came from nowhere?
17:22What happens when the when a fact comes from nowhere?
17:25And how do I actually handle these kinds of of hallucinations?
17:29So, this whole platform is actually to make a very simple statement very obvious, which is we can have the smartest models in the world.
17:39We can hit some crazy cool piece of technology, but we still have the same database problems that we've had from the beginning, which are I cannot if I search through a
17:51a row and that column does not value doesn't exist, I won't get a retrieval back.
17:56We're now out of the search phase of AI.
17:58We need to be building infrastructure.
18:00We need to be building scaffolding and harnesses and substrates or whatever the cool VC term is these days that actually enable us to make use of this technology.
18:09So, this is my challenge and my ask to all of you.
18:11Um I hope you go out there and build something compelling, but I hope you go build something compelling that you can trust.
18:18Thank you.
18:19[applause]
18:27Hey.