WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=Tt2kX2sgQio

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/Tt2kX2sgQio.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.560 --> 00:00:14.360
Hey everyone, thank you for being here.

s3
00:00:14.360 --> 00:00:18.640
My name is Venu Ganesh and I'm the CEO and co-founder of Kepler.

s4
00:00:18.640 --> 00:00:23.920
Today, I'm going to talk to you about how we built verifiable AI for financial services.

s5
00:00:24.200 --> 00:00:25.720
First, a little about me.

s6
00:00:25.720 --> 00:00:33.080
My career has been working in fairly difficult places to work in terms of numerical accuracy and verifiability.

s7
00:00:33.080 --> 00:00:38.000
Began my career at Palantir where I led the compute platform as well as a lot of our USG engagements.

s8
00:00:38.000 --> 00:00:40.960
Built and sold a alt data startup.

s9
00:00:40.960 --> 00:00:43.040
Then was head of business engineering at Citadel.

s10
00:00:43.040 --> 00:00:46.000
I have some Citadel colleagues here in the audience as well.

s11
00:00:46.000 --> 00:00:51.880
I've advised a bunch of startups and been lucky that all of them have reached pretty positive outcomes.

s12
00:00:51.880 --> 00:00:56.760
So this whole talk is a distilled version of a case study Anthropic did on Kepler.

s13
00:00:56.760 --> 00:01:03.120
If you scan that QR code, we're the only company they've ever done a case study of, which is kind of cool.

s14
00:01:03.120 --> 00:01:05.600
And so this is a distilled version of that.

s15
00:01:05.600 --> 00:01:15.680
That has a lot more information about how we were able to do what we do, why it's been pretty impactful in financial services, and really where we go from here.

s16
00:01:15.680 --> 00:01:24.760
So my only goal with this whole talk is to convince you that AI is going to start doing some very powerful things in terms of producing work product.

s17
00:01:24.760 --> 00:01:27.840
So everything that I tell you is inevitable.

s18
00:01:27.840 --> 00:01:33.320
Whether it's Kepler or whether it's anyone else, we are already on this journey and this trajectory.

s19
00:01:33.320 --> 00:01:36.600
So we all better be ready.

s20
00:01:36.600 --> 00:01:43.840
So I first want to observe every talk that we've seen in the financial services space so far has been about producing more.

s21
00:01:43.840 --> 00:01:45.160
Like how do I token max?

s22
00:01:45.160 --> 00:01:47.320
How do I get AI to do more?

s23
00:01:47.320 --> 00:01:54.080
Very little of it is about how to actually make AI trustworthy or produce trustworthy products.

s24
00:01:54.080 --> 00:01:59.400
And what's interesting is even terms like trust and verify-ability have actually been largely abused.

s25
00:01:59.400 --> 00:02:01.600
Evals are not verifiable.

s26
00:02:01.600 --> 00:02:06.880
You cannot take a non-deterministic LLM and eval your way to something deterministic.

s27
00:02:06.880 --> 00:02:08.960
These are probability machines.

s28
00:02:08.960 --> 00:02:15.640
And the kind of underlying reason for this is that AI has made a writing problem a reading problem.

s29
00:02:15.640 --> 00:02:26.080
We can produce insane amounts of content, whether that's code, whether it's marketing, whether it's like a DCF in record time.

s30
00:02:26.080 --> 00:02:27.520
But we can't easily verify this.

s31
00:02:27.520 --> 00:02:33.440
And that's because for years the hardest part about this whole process was actually producing the work.

s32
00:02:33.440 --> 00:02:43.920
Edge and alpha came from people like Citadel being able to hire hundreds of analysts who could scour the internet and understand where any source of alpha could exist.

s33
00:02:43.920 --> 00:02:48.400
So it really came from this idea of being able to consume content.

s34
00:02:48.400 --> 00:02:54.720
The problem is when a model reads everything, you have the most real version of alpha decay that you possibly can.

s35
00:02:54.720 --> 00:02:59.120
There is no edge if everyone can look at Tegus and get all the same information.

s36
00:02:59.120 --> 00:03:04.160
So the hard part now is trusting what actually got produced by the model.

s37
00:03:04.440 --> 00:03:05.160
And this is kind of funny.

s38
00:03:05.160 --> 00:03:08.480
This is not necessarily a finance problem.

s39
00:03:08.480 --> 00:03:12.080
Every system that exists has some form of this.

s40
00:03:12.080 --> 00:03:15.160
Software, we run CICD.

s41
00:03:15.160 --> 00:03:16.400
We do unit tests.

s42
00:03:16.400 --> 00:03:17.840
We do integration tests.

s43
00:03:17.840 --> 00:03:19.320
We have code reviews.

s44
00:03:19.320 --> 00:03:23.320
When a doctor writes a prescription, a pharmacist fills that prescription.

s45
00:03:23.320 --> 00:03:28.519
So if it says 10,000 milligrams of a medication, someone catches that.

s46
00:03:28.519 --> 00:03:30.080
We have a pilot and a co-pilot.

s47
00:03:30.080 --> 00:03:33.519
We have an EMT that's a primary EMT and a secondary.

s48
00:03:33.519 --> 00:03:40.200
In finance, we have maybe an overworked VP as a verification layer, but that concept doesn't really exist.

s49
00:03:40.200 --> 00:03:42.840
And now I'm going to say something even more aggressive.

s50
00:03:42.840 --> 00:03:49.480
Uh the reason that people buy products like Bloomberg and FactSet is to displace culpability.

s51
00:03:49.480 --> 00:03:52.240
When you buy a tool like that, you know that information free.

s52
00:03:52.240 --> 00:03:54.200
It exists in SEC filings.

s53
00:03:54.200 --> 00:04:06.600
But, you believe that because a bunch of contractors or folks overseas vetted this data and stuck it in a central instance, at least if it's wrong, everyone on Wall Street has the same incorrect information.

s54
00:04:06.600 --> 00:04:08.560
And that's interesting in certain ways.

s55
00:04:08.560 --> 00:04:11.120
It's also kind of a scary proposition.

s56
00:04:11.120 --> 00:04:14.760
And that's because every one of these tools is read-only.

s57
00:04:14.760 --> 00:04:21.680
You as an analyst look at Bloomberg, you look at FactSet, you consume information, and you produce the work product.

s58
00:04:21.680 --> 00:04:28.520
And that's where the biggest gap and biggest wall to real meaningful adoption across Wall Street has actually been.

s59
00:04:28.520 --> 00:04:30.919
And so, why does this actually matter?

s60
00:04:30.919 --> 00:04:33.919
We can't use AI properly in this ecosystem.

s61
00:04:33.919 --> 00:04:42.240
There is a reason that analysts are still working till 4:00 a.m., and there's a reason folks in investment banks are actually dying because of the hours they're putting in.

s62
00:04:42.240 --> 00:04:51.120
Because AI can produce a very confident answer, but when it comes to producing any kind of meaningful work product, we're totally lacking.

s63
00:04:51.120 --> 00:05:00.040
Not only that, we have the SEC, we have the OCC, we have a number of these regulatory agencies who exist to make sure that you are not insider trading,

s64
00:05:00.040 --> 00:05:13.560
or you're not producing a trading decision that can't be justified or backed by some set of primitives or some set of information that you can reliably say, "I made this decision because of these sources of information."

s65
00:05:13.560 --> 00:05:26.400
And so, in this ecosystem, the biggest challenge is how do I get AI to jump from producing the search technologies that it's doing right now to producing work product.

s66
00:05:26.400 --> 00:05:39.320
That can mean a fairness opinion, it can mean a DCF, it can mean an investment memo, it can mean looking at every SIM that your firm had 5 years ago and figuring out why your IRR number wasn't what it should have been, or anything else.

s67
00:05:39.320 --> 00:05:43.880
And so, the whole industry has solved this by citing things.

s68
00:05:43.880 --> 00:05:50.640
When you search the internet with Claude or ChatGPT, it gives you a list of sources that pulled information from.

s69
00:05:50.640 --> 00:05:54.400
What's ironic is you can't easily curate those sources.

s70
00:05:54.400 --> 00:06:02.840
So, if you find my random Substack that says Palantir is going to be $3,000 a share, and you trade off of that, please go do that because it'll be very helpful for me.

s71
00:06:02.840 --> 00:06:05.720
But, that's not a real vetted source.

s72
00:06:05.720 --> 00:06:11.880
Seeking Alpha and some of these blogs or Reddit posts, they're data points, but they're not real vetted sources.

s73
00:06:11.880 --> 00:06:16.240
So, showing where you got the information from is only half the battle.

s74
00:06:16.240 --> 00:06:18.160
And that's where we're limited right now.

s75
00:06:18.160 --> 00:06:21.840
The citation is effectively an after-the-fact audit.

s76
00:06:21.840 --> 00:06:32.960
Now, a verification is a deterministic, repeatable, numerically verifiable mechanism that we can use to produce validity that a number is right.

s77
00:06:32.960 --> 00:06:34.520
That was a lot of buzzwords.

s78
00:06:34.520 --> 00:06:39.480
A verification just means I can prove deterministically this number is right.

s79
00:06:39.480 --> 00:06:47.760
So, when I extract a revenue number from a 10-K, I can deterministically prove that is the correct number from that 10-K.

s80
00:06:47.760 --> 00:06:50.600
So, these are two sides of kind of the same game.

s81
00:06:50.600 --> 00:06:57.760
One is showing where you got the sources from, and one is verifying that the information that you pulled out is actually correct.

s82
00:06:57.760 --> 00:06:59.760
This becomes challenging.

s83
00:06:59.760 --> 00:07:05.440
It becomes challenging because verification is not a outcome.

s84
00:07:05.440 --> 00:07:15.120
It lives in the path or the set of steps that are required for you to produce information that is valid for your individual firm.

s85
00:07:15.120 --> 00:07:24.800
Finance is one of the rare industries where two people can be have the same information, and one can be long a stock, and one can be short that stock with the exact same data.

s86
00:07:24.800 --> 00:07:28.480
And so, the idea of verification is not actually ground truth.

s87
00:07:28.480 --> 00:07:35.640
It is verifying that you got an output that respects the nouns and verbs or the rules of your organization.

s88
00:07:35.640 --> 00:07:45.440
If a desk at Citadel, like a TMT desk at Citadel, believes that a particular stock is going to go up into the right, another TMT desk at Citadel may have the exact opposite belief,

s89
00:07:45.440 --> 00:07:50.480
and they may have the same verification mechanisms that produce vastly different outcomes.

s90
00:07:50.480 --> 00:07:53.240
And so, this really comes on to something simple.

s91
00:07:53.240 --> 00:07:55.000
What are the sources that we trust?

s92
00:07:55.000 --> 00:08:00.200
What are the transformations that we apply on those sources to produce information that we care about?

s93
00:08:00.200 --> 00:08:04.400
And how do we make sure that's codified in a way that makes logical sense?

s94
00:08:04.400 --> 00:08:07.440
And so, the whole point of this is simple.

s95
00:08:07.440 --> 00:08:12.440
This came from a 10K is not the the validation as a whole.

s96
00:08:12.440 --> 00:08:18.520
Your job now is to figure out as an individual, how do I use AI in a verifiable way?

s97
00:08:18.520 --> 00:08:19.960
And here's the honest answer.

s98
00:08:19.960 --> 00:08:21.360
You don't.

s99
00:08:21.360 --> 00:08:24.560
AI is great at doing non-deterministic tasks.

s100
00:08:24.560 --> 00:08:28.080
It can solve problems in a way that's novel.

s101
00:08:28.080 --> 00:08:35.080
It can figure out exactly how to do EBITDA adjustments, but it can't be the one responsible for doing the mathematical adjustments.

s102
00:08:35.080 --> 00:08:37.080
Because it's a probability machine.

s103
00:08:37.080 --> 00:08:39.800
It is great at next token prediction.

s104
00:08:39.800 --> 00:08:51.560
So, the second contention of this whole talk is that you cannot use AI to produce verifiable work product in finance without augmenting it with a deterministic substrate.

s105
00:08:51.560 --> 00:09:01.240
Which effectively means, if you're a portfolio manager at Citadel, you have access to a number of deterministic tools that you use to make your trading decisions.

s106
00:09:01.240 --> 00:09:03.839
We need to model AI like that PM.

s107
00:09:03.839 --> 00:09:10.320
It needs to exist in grounding, needs to exist with a certain risk threshold and verifiability.

s108
00:09:10.320 --> 00:09:11.920
So, here's how we did it.

s109
00:09:11.920 --> 00:09:13.920
Here's what Anthropic was excited about.

s110
00:09:13.920 --> 00:09:26.640
We have three tenants that we use to ensure that numerical accuracy is a tenant of the Kepler platform, and that allows us to produce pretty powerful, pretty verifiably reliable information.

s111
00:09:26.640 --> 00:09:29.520
The first is atomic provenance, and I'll talk through all of these.

s112
00:09:29.520 --> 00:09:36.280
The second is scope determinism, and the third is derivation chains, uh which we'll all talk about.

s113
00:09:36.280 --> 00:09:38.280
But the core premise is this.

s114
00:09:38.280 --> 00:09:40.960
There are certain things humans should never do.

s115
00:09:40.960 --> 00:09:54.200
And I will tell you right now, I don't believe a human should sit there and look at a PDF 10K, 10Q, 8K, or earnings call transcript and on one screen take that number and put it into an Excel model on the other screen.

s116
00:09:54.200 --> 00:09:55.800
Humans were not built for that.

s117
00:09:55.800 --> 00:10:00.760
That's not like there's a reason we don't have databases just written in our heads.

s118
00:10:00.760 --> 00:10:06.120
And so, let's break this down for how you can actually use AI to produce work product.

s119
00:10:06.120 --> 00:10:07.800
Let's talk about provenance.

s120
00:10:07.800 --> 00:10:12.320
Provenance as a whole just means writing down exactly where you got the information.

s121
00:10:12.320 --> 00:10:20.480
Now, uh have a lot of respect for everyone else who got here on stage, but we did a model we trained a model that was really good at extracting information.

s122
00:10:20.480 --> 00:10:23.600
It outperformed foundation models, it's 94% great.

s123
00:10:23.600 --> 00:10:25.320
It's in the article.

s124
00:10:25.320 --> 00:10:29.320
Who here would trade off of something that's 94% accurate?

s125
00:10:29.320 --> 00:10:29.920
Right.

s126
00:10:29.920 --> 00:10:38.920
So, fine-tuning your way on probabilistic solutions still is it's really cool and like TechCrunch will be really excited about it, but like no one else really cares about it.

s127
00:10:38.920 --> 00:10:45.200
And that's because a wrong number is still wrong if you're in that unfortunate 6%.

s128
00:10:45.200 --> 00:10:51.360
So, with atomic provenance, what we do is the model writes effectively a reference to the number.

s129
00:10:51.360 --> 00:10:54.240
It cannot write the number or manipulate the number in any way.

s130
00:10:54.240 --> 00:10:56.800
It doesn't even understand what that number is.

s131
00:10:56.800 --> 00:10:59.440
We have tools that are really good at understanding numbers.

s132
00:10:59.440 --> 00:11:00.760
They're databases.

s133
00:11:00.760 --> 00:11:07.640
They are systems that can codify information and read them and write them with appropriate fidelity.

s134
00:11:07.640 --> 00:11:09.560
So, that's the first piece of how we do things.

s135
00:11:09.560 --> 00:11:19.440
Anytime a model makes a decision, it makes the decision to figure out exactly where it got the number from and hand off to something that can write that number.

s136
00:11:19.440 --> 00:11:28.040
We then run it through a deterministic check where any kind of a wrong number, if we can't verify it independently, we strip it out.

s137
00:11:28.040 --> 00:11:39.320
That number will never make it to someone if it doesn't follow the deterministic check, the Providence ledger, and most importantly, uh the whole cycle kind of repeating at least a couple of times.

s138
00:11:39.320 --> 00:11:46.960
So, this is not me saying, "Have 10 models and each individual have OpenAI check chat check Anthropic and have Anthropic check XAI."

s139
00:11:46.960 --> 00:11:49.960
These are not probabilistic systems evaluating each other's work.

s140
00:11:49.960 --> 00:11:57.840
There's a core canonical process of extracting a number, persisting it, and making sure that process actually occurred properly.

s141
00:11:58.200 --> 00:12:02.080
And this is when I say atomic traditional atomicity in like database land.

s142
00:12:02.080 --> 00:12:04.080
Second, scope determinism.

s143
00:12:04.080 --> 00:12:10.200
This is This was a super controversial idea like a year ago, and VCs were like, "This is crazy."

s144
00:12:10.200 --> 00:12:13.920
Now, this half of this talk has been This session has been about this.

s145
00:12:13.920 --> 00:12:16.800
Um the model is really good at reasoning and planning.

s146
00:12:16.800 --> 00:12:18.240
Intelligence is commoditized.

s147
00:12:18.240 --> 00:12:19.680
GLM 52 shows it.

s148
00:12:19.680 --> 00:12:22.120
You can download it off of Hugging Face right now.

s149
00:12:22.120 --> 00:12:24.840
You have something as powerful as Opus 48.

s150
00:12:24.840 --> 00:12:28.720
Now, what the model cannot do is math.

s151
00:12:28.720 --> 00:12:29.600
And why would it?

s152
00:12:29.600 --> 00:12:34.920
Why would I run 1 + 1 through a multi-billion parameter model instead of one CPU cycle?

s153
00:12:34.920 --> 00:12:39.808
Unless you're companies that are giving bonuses on people token maxing, which is another Polarys thing.

s154
00:12:39.808 --> 00:12:39.880
[snorts]

s155
00:12:39.880 --> 00:12:44.760
Um and so, what the model does is the model decides what to compute.

s156
00:12:44.760 --> 00:12:47.040
It never does the computation itself.

s157
00:12:47.040 --> 00:12:58.000
And so, from the Kepler platform perspective, what we do is we split the deterministic pieces of the model, which are none, from the non-deterministic pieces of the model, which are all of them.

s158
00:12:58.000 --> 00:13:02.760
And we give the model the right tooling and technology to calculate the deterministic pieces.

s159
00:13:02.760 --> 00:13:04.280
Now, let's get really concrete.

s160
00:13:04.280 --> 00:13:09.200
A model can read something like, "Okay, I need to understand what net margin is.

s161
00:13:09.200 --> 00:13:18.560
I know the right pieces of information to go to to get that data, but I can't be the entity that's actually running the code behind the scenes to pull that number out of a PDF

s162
00:13:18.560 --> 00:13:22.080
or parsing the XBRL behind the scenes to pull that information out.

s163
00:13:22.080 --> 00:13:23.520
That is code."

s164
00:13:23.520 --> 00:13:30.920
Now, the deterministic pieces of the platform pull that information out and persist it outside of anything the model understands.

s165
00:13:30.920 --> 00:13:43.800
And with those two together, we can actually produce a numerically accurate answer to the question of like, what was the net What was the net margin of this stock or this company last quarter?

s166
00:13:43.800 --> 00:13:49.000
This also is a lot cheaper because again, I don't need the model to do a bunch of stuff it shouldn't be doing.

s167
00:13:49.560 --> 00:13:54.040
Now, the last piece here is how do we actually do reconciliation?

s168
00:13:54.040 --> 00:14:06.160
When someone asks about a ratio like a gross margin or anything else that doesn't exist in a filing, therefore, I can't just go look up what the gross margin is or what the set of EBITDA adjustments were.

s169
00:14:06.160 --> 00:14:11.480
The other thing that's really complicated is everyone calculates these ratios and these multiples differently.

s170
00:14:11.480 --> 00:14:14.160
Everyone does enterprise value calculations differently.

s171
00:14:14.160 --> 00:14:18.160
Things that are considered recurring or non-recurring may be unique.

s172
00:14:18.160 --> 00:14:29.120
So, not only do we have to codify that in the processes that are run, but we need some kind of a chain of events to figure out what went into producing an individual number and what went into producing an outcome.

s173
00:14:29.120 --> 00:14:37.160
This is not any different than the chain of events that an analyst does on a desk at a hedge fund to make a risk-reward or a trading decision.

s174
00:14:37.160 --> 00:14:41.839
It's just done by the model in a way that can be replayed and rewound.

s175
00:14:41.839 --> 00:14:44.920
And so, that leads us to something fairly simple here.

s176
00:14:44.920 --> 00:14:57.800
Uh we have a system that knows which data points it's allowed to produce, meaning from structured filings, from numerical data, from any other ecosystem, it knows what it's allowed to produce,

s177
00:14:57.800 --> 00:14:59.640
and it knows what it's never allowed to produce.

s178
00:14:59.640 --> 00:15:05.320
So, if it's pulling things out of prose, raw tables, or anything else, it doesn't do that extraction.

s179
00:15:05.320 --> 00:15:15.880
What this allows us to do is this allows us to do things like consolidate financial statements in seconds with every number tied back to its individual source.

s180
00:15:15.880 --> 00:15:24.720
Which means not picking any companies, but the CapIQ's, the Deloopa's that are all using contractors for this, we don't need to do that anymore.

s181
00:15:24.720 --> 00:15:31.040
We can actually create a financial model in a numerically accurate way that allows you to build work product.

s182
00:15:31.040 --> 00:15:33.840
We can build a DCF in the format that you actually want.

s183
00:15:33.840 --> 00:15:38.720
In knowledge, the model will not hallucinate that a row exists that shouldn't exist.

s184
00:15:38.720 --> 00:15:46.920
Now, the really crazy thing and why I say this is inevitable is everything that I'm telling you generally generalizes past finance.

s185
00:15:46.920 --> 00:15:50.400
We're picking numbers here because finance cares about numbers.

s186
00:15:50.400 --> 00:15:56.760
But you can imagine a world where every court case, if you're a Harvey or a Legora, runs through the same process.

s187
00:15:56.760 --> 00:16:07.080
A pre-processing step that understands that we can extract entities like case A versus case B and store that deterministically so we don't hallucinate citations.

s188
00:16:07.080 --> 00:16:15.480
Or every drug discovery formulation that exists in NIH white papers such that we never miss a compound or anything else.

s189
00:16:15.480 --> 00:16:27.920
And so, the kind of uh interesting dimension that we're entering is we're an ecosystem right now where we're almost like pre-SSL in the e-commerce ecosystem.

s190
00:16:27.920 --> 00:16:30.040
Where like what's the TAM of e-commerce?

s191
00:16:30.040 --> 00:16:35.920
Trillions, but how many people were comfortable putting their credit card number on the internet before there was security?

s192
00:16:35.920 --> 00:16:37.360
Zero.

s193
00:16:37.360 --> 00:16:39.520
So, we're in the last step.

s194
00:16:39.520 --> 00:16:44.080
AI can now produce verifiable work product across a number of industries.

s195
00:16:44.080 --> 00:16:53.839
Meaning the rag platforms of the past are really, really helpful and really cool in codifying workflows, but there's a reason that the foundational labs are going after every one of these.

s196
00:16:53.839 --> 00:17:00.360
There's a reason that Claude for science is not a deterministic system, but still a rag-based system.

s197
00:17:00.360 --> 00:17:12.000
And so, the piece that should be really exciting about this whole thing is there is a piece of this that no one has built yet in a variety of disciplines and a variety of industries.

s198
00:17:12.000 --> 00:17:14.319
We're really good at consuming tokens.

s199
00:17:14.319 --> 00:17:18.480
In fact, there's a club here for people that consumed a billion plus tokens.

s200
00:17:18.480 --> 00:17:19.880
They're walking around with gold cards.

s201
00:17:19.880 --> 00:17:21.560
It's kind of funny, actually.

s202
00:17:21.560 --> 00:17:24.079
But, like, that's kind of hilarious, right?

s203
00:17:24.079 --> 00:17:31.600
Like, in what time in history has an employee been rewarded for your company to pay another vendor for how much money you're spending?

s204
00:17:31.600 --> 00:17:39.560
And so, token maxing, I think, is now being thought of as not the right approach here to actually solve your problems.

s205
00:17:39.560 --> 00:17:46.720
So, the natural thing will become a rush to the bottom, which is an optimization problem, which we've seen over and over again.

s206
00:17:46.720 --> 00:17:59.200
If you rewind time, I actually sat on this stage 4 years ago, maybe not this particular room, talking about like Snowflake Summit and Databricks Summit, where the idea was all of a sudden people wanted ROI

s207
00:17:59.200 --> 00:18:04.560
on top of like to understand the ROI of their Snowflake investment or their Databricks investment.

s208
00:18:04.560 --> 00:18:06.960
And we started doing cost optimization.

s209
00:18:06.960 --> 00:18:15.680
We started figuring out how to make sure every dollar of capital we put into Snowflake and into Databricks went to actually producing a pipeline that people were using.

s210
00:18:15.680 --> 00:18:17.560
That same trend is about to start.

s211
00:18:17.560 --> 00:18:21.680
And we're figuring out right now, how do we use the right tool for the right job?

s212
00:18:21.680 --> 00:18:27.520
And sometimes you don't need a multi-billion parameter model when one CPU cycle will just work.

s213
00:18:27.520 --> 00:18:35.880
So, kind of wrapping this up, AI has made producing work completely, I say nearly free, but like, thank you VCs, heavily subsidized.

s214
00:18:35.880 --> 00:18:39.280
The reading problem is still very open.

s215
00:18:39.280 --> 00:18:44.680
And you verifying a data point is not enough anymore.

s216
00:18:44.680 --> 00:18:55.080
The system has to be able to track or track its own provenance and actually ensure that the numbers that you're pulling represent your own unique company philosophies.

s217
00:18:55.080 --> 00:19:05.760
Citations got us like 50% of the way there, but the next half of the verifiability and provability is going to be how we start using this in real like valuable,

s218
00:19:05.760 --> 00:19:07.960
verifiable work.

s219
00:19:07.960 --> 00:19:12.320
And the interesting thing here is the work product itself is the proof.

s220
00:19:12.320 --> 00:19:21.120
In code, we have I mean unit tests, we have every single pull request and every commit and every code review on that pull request stored in perpetuity.

s221
00:19:21.120 --> 00:19:29.840
There are companies here trying to mine that information to create a representation of your on your ontology right now on a company specific basis.

s222
00:19:29.840 --> 00:19:32.200
We need that same ecosystem in finance.

s223
00:19:32.200 --> 00:19:35.880
So, let's just say this, all these problems are solved at this point.

s224
00:19:35.880 --> 00:19:39.080
The last remaining mile is going to be that personalization.

s225
00:19:39.080 --> 00:19:49.880
So, the second version of this in 2027 hopefully one of us will be on stage talking about how we're now able to build verifiable ontologies that actually proxy our investment processes

s226
00:19:49.880 --> 00:19:56.520
instead of saying how do I not spend a trillion tokens to solve this individual problem.

s227
00:19:56.520 --> 00:20:00.600
Um so, we're also I mean Susanna, we're both from Kepler.

s228
00:20:00.600 --> 00:20:02.080
We're growing pretty quickly.

s229
00:20:02.080 --> 00:20:05.600
Obligatory come join us if these problems are interesting to you.

s230
00:20:05.600 --> 00:20:07.720
And yeah, I think we're starting in finance now.

s231
00:20:07.720 --> 00:20:10.080
We'll be in a lot of different dimensions pretty quickly.

s232
00:20:10.080 --> 00:20:12.418
Um so, thank you so much and happy to answer any questions.

s233
00:20:12.418 --> 00:20:14.418
[applause]

s234
00:20:17.003 --> 00:20:19.003
[applause]

s235
00:20:19.880 --> 00:20:22.120
Yeah, go ahead.

s236
00:20:27.480 --> 00:20:31.680
Yes, your question is about provenance and from the provenance ledger, where does the number actually come from?

s237
00:20:31.680 --> 00:20:34.560
At its core, the number comes from three different things.

s238
00:20:34.560 --> 00:20:37.160
It comes from extracted information from the filings.

s239
00:20:37.160 --> 00:20:43.360
It comes from either a mathematical calculation that we do to derivative production, so like a ratio or something else.

s240
00:20:43.360 --> 00:20:52.360
Or it comes from your internal documents or other internal pieces of information that operate in conjunction with that external data to produce that individual data point.

s241
00:20:52.360 --> 00:21:02.360
So, anytime the model effectively is responsible for telling some entity to do an IO operation, that's part of the provenance chain.

s242
00:21:02.400 --> 00:21:04.400
Yeah.

s243
00:21:08.320 --> 00:21:10.320
What are your customers most excited about in your product?

s244
00:21:10.320 --> 00:21:13.320
Yeah, so the question is um what are our customers most excited about?

s245
00:21:13.320 --> 00:21:13.840
It's funny.

s246
00:21:13.840 --> 00:21:19.600
Uh everyone wants AI like the dream is the AI portfolio manager.

s247
00:21:19.600 --> 00:21:22.640
The portfolio managers don't want the AI portfolio manager.

s248
00:21:22.640 --> 00:21:24.440
Like they want the AI analyst.

s249
00:21:24.440 --> 00:21:33.680
And so the thing they're most excited about is a way of rapidly producing, rapidly doing kind of the repeatable painful tasks that their analysts are doing.

s250
00:21:33.680 --> 00:21:38.160
Things like uh analyst actually sits there and listens to an earnings call transcript.

s251
00:21:38.160 --> 00:21:41.480
If we didn't have to have an analyst do that, it would be amazing.

s252
00:21:41.480 --> 00:21:51.640
Or an analyst sits there with like 15 tabs and they're opening every 8-K and 10-K over the last, you know, whatever years to create the V0 of a financial model.

s253
00:21:51.640 --> 00:21:53.960
So they're most excited about getting their analyst time back.

s254
00:21:53.960 --> 00:21:56.440
That's the honest answer.

s255
00:21:56.440 --> 00:21:57.800
And these are expensive analysts.

s256
00:21:57.800 --> 00:22:03.240
Like these Some of these folks make 6-700k a year to do this.

s257
00:22:05.000 --> 00:22:05.280
Cool.

s258
00:22:05.280 --> 00:22:09.880
I'm happy to answer any more questions outside as actually Yeah, outside as well if you want to defer it time.

s259
00:22:09.880 --> 00:22:12.560
Yeah, the gentleman in the back.

s260
00:22:26.695 --> 00:22:28.695
[music]