WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=cf2IhzqeQH4

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/cf2IhzqeQH4.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:12.880 --> 00:00:13.480
Yeah, we're good.

s2
00:00:13.480 --> 00:00:14.560
Okay.

s3
00:00:14.560 --> 00:00:16.080
So, hi everyone.

s4
00:00:16.080 --> 00:00:19.840
Given there's I hear there's a match right now as we speak.

s5
00:00:19.840 --> 00:00:23.200
So, given that you are here means a lot to me.

s6
00:00:23.200 --> 00:00:25.480
So, at least I'm I'm winning the World Cup right now.

s7
00:00:25.480 --> 00:00:27.600
So, that's that's a that's a big thing.

s8
00:00:27.600 --> 00:00:28.560
Thank you for joining me.

s9
00:00:28.560 --> 00:00:31.760
I flew all the way over on co-founder CEO of Stig.

s10
00:00:31.760 --> 00:00:41.960
I flew all the way over to make a kind of a funky statement on an AI conference which is literally every AI company right now is accidentally building a bank.

s11
00:00:41.960 --> 00:00:53.880
And when I mean bank, I'm going to try to kind of unpack this thing not as like a weird marketing metaphor, but effectively what what is really happening right now with the AI economy

s12
00:00:53.880 --> 00:01:06.840
and why a lot of the constructs and the things that we are seeing using consuming and paying for in AI are actually really behaving like constructs that we know from the banking financial systems.

s13
00:01:06.840 --> 00:01:07.200
So, cool.

s14
00:01:07.200 --> 00:01:09.680
Thank you for for for having me.

s15
00:01:09.680 --> 00:01:13.040
Um and let's get it going.

s16
00:01:13.360 --> 00:01:14.760
By the way, I'm the first talk today.

s17
00:01:14.760 --> 00:01:15.920
So, the clicker is not working.

s18
00:01:15.920 --> 00:01:18.880
It's going to be interesting here and out as they move for this.

s19
00:01:18.880 --> 00:01:23.360
So, let me take you through a little bit of background.

s20
00:01:23.360 --> 00:01:39.760
As I was working through this talk back in April really every single code generating product platform out there just broke under this new type of economy and under this new type of

s21
00:01:39.760 --> 00:01:41.560
consumer behavior.

s22
00:01:41.560 --> 00:01:48.200
And in just a you know, matters like 5 weeks everything felt like pricing emergency one after the other.

s23
00:01:48.200 --> 00:01:56.160
And the case I want to make here is that these pricing emergencies go way beyond just emergencies from a financial commercial set of the house.

s24
00:01:56.160 --> 00:02:04.000
These really are infrastructure emergencies that are emerging because of how those systems were built to begin with.

s25
00:02:04.000 --> 00:02:15.640
So, just to give you some perspective, we had in in April we had Anthropic basically eliminating access for Open Clo and other third-party agents to basically use the subscriptions plans.

s26
00:02:15.640 --> 00:02:26.640
Then immediately we had OpenAI basically changing how the product here is being priced, bumping the price overnight 5 5x, which is quite intense.

s27
00:02:26.640 --> 00:02:35.560
And then we had GitHub freezing and then completely eliminating access trial access free access to their We have some echo here.

s28
00:02:35.560 --> 00:02:39.760
To their free plans for for for Copilot.

s29
00:02:39.760 --> 00:02:53.760
And effectively like all these companies all at once in a single month experienced what it means to actually scale on scale AI and what actually happens when you don't have the fundamentals in place.

s30
00:02:53.840 --> 00:02:56.959
So, let me take one one step farther.

s31
00:02:56.959 --> 00:02:59.760
So, let's zoom into even just one of the use cases, right?

s32
00:02:59.760 --> 00:03:06.280
So, if you think about Anthropic and you think about the Open Clo use case, right?

s33
00:03:06.280 --> 00:03:15.640
What happened was effectively Anthropic was subsidizing every consumption of every user using Open Clo within the Clo Clo Max subscription.

s34
00:03:15.640 --> 00:03:20.840
So, effectively what the customers were paying daily was, you know, dollars and cents.

s35
00:03:20.840 --> 00:03:35.239
But on the on the cost side for Anthropic, it was like 150s to even 750 subsidized, which is effectively not an economics that you can scale even for the company at at the size at the scale of Anthropic,

s36
00:03:35.239 --> 00:03:35.920
right?

s37
00:03:35.920 --> 00:03:47.720
So, what it seemed to be a business problem really was an infrastructure problem because Anthropic didn't have a an effective way to delineate between users who are using their APIs

s38
00:03:47.720 --> 00:03:51.160
and users who are using their subscription um, uh, programs.

s39
00:03:51.160 --> 00:03:54.400
And so, they had to immediately cease and stop access, right?

s40
00:03:54.400 --> 00:03:56.600
They had to to to react.

s41
00:03:56.600 --> 00:03:58.800
Um, cool.

s42
00:03:58.800 --> 00:04:04.600
So, and this is not unique to Anthropic, guys.

s43
00:04:04.600 --> 00:04:13.760
Like, uh, we're seeing this like, I don't know if you all heard read the news most recently with a company, uh, undisclosed company burning through like half a billion

s44
00:04:13.760 --> 00:04:21.959
of their AI, uh, cloud credits, um, just merely having like a few employees burning through the entire organization contract.

s45
00:04:21.959 --> 00:04:32.800
Uh, we had, uh, Uber saying they're basically torched down their entire, uh, AI usage consumption budget for the year in just a few weeks into the year.

s46
00:04:32.800 --> 00:04:44.680
And then we had Rep- Replit giving an example of what could happen if, uh, a single org has three users that basically burning down the entire credit pool for Replit for the entire org.

s47
00:04:44.680 --> 00:04:50.800
So, really what happening here is, uh, this is not just an an an Anthropic problem.

s48
00:04:50.800 --> 00:05:01.840
And I think what's what's mutual and kind of like switching gears from the examples to the actual what's really happening here, um, the problem basically with with all these three examples

s49
00:05:01.840 --> 00:05:14.600
is that the checks for what you're entitled, what you're allowed to do, whether you're an agent or a user, we are AI product happened, uh, after, uh, the invoice and not before.

s50
00:05:14.600 --> 00:05:23.919
So, basically with each and every one of those examples, you were allowed to consume, you were allowed to use the product, you were allowed to burn tokens, and checks happen only after the invoice.

s51
00:05:23.919 --> 00:05:36.800
So, basically the spend and the aftereffect gets reconciled only after it's too late, only after the invoice is is sent, and this is effectively bad for business, it's sticker shock, it's bad for

s52
00:05:36.800 --> 00:05:38.480
practice of how we use these systems.

s53
00:05:38.480 --> 00:05:52.880
These systems are now mission critical, and we're basically none of these companies had in place the financial infrastructure that allows, exactly like banks, guys, to check before we draw down, before we consume, right?

s54
00:05:52.880 --> 00:06:02.520
So, I think what what we're seeing is effectively uh really is a change in how we architect, and how we ship, and how we build software.

s55
00:06:02.520 --> 00:06:08.200
AI is not just changing what is value, is not just changing how we interact, who is the users.

s56
00:06:08.200 --> 00:06:27.240
It also effectively changing what exactly is getting paid, and how we actually architect for systems that can uh handle runtime, handle the complexity of usage happening before the the the invoice, before the checks, and how we actually can architect for that.

s57
00:06:27.240 --> 00:06:36.760
So, I think what's interesting, and what you see in front of you um is basically on the left side, what we're seeing is how most teams ship this architecture today,

s58
00:06:36.760 --> 00:06:44.360
and you can see that basically the check and the settlements are happening after after the inference, after the usage is being logged.

s59
00:06:44.360 --> 00:06:46.000
And that's typically too late.

s60
00:06:46.000 --> 00:06:56.640
We see situations like spikes, overspends, overages, like experiences that are bad for business, bad for your users, and quite frankly, in many cases, also bad for the vendor.

s61
00:06:56.640 --> 00:07:05.600
Because if you didn't allocate enough resources and compute to actually allow for this to happen, you you also have a margin problem potentially, right?

s62
00:07:05.600 --> 00:07:16.200
On the right side, we're basically saying, or I'm trying to make the case for you all that with AI, with the current infrastructure, and how AI is being built and sold,

s63
00:07:16.200 --> 00:07:28.000
we actually need to enforce and and do this synchronously before we actually update the balances, and reconcile and settle in in async way after after effects.

s64
00:07:28.000 --> 00:07:34.600
So, hot path actually get checked synchronously, and everything that comes after uh is basically reconciled after.

s65
00:07:34.600 --> 00:07:39.240
I think the best example, guys, is if imagine you go to the ATM, right?

s66
00:07:39.240 --> 00:07:40.520
And you draw cash.

s67
00:07:40.520 --> 00:07:46.000
When you draw the cash, it that the check if you are allowed to draw it, doesn't happen after you get the cash.

s68
00:07:46.000 --> 00:07:47.880
It happens before you draw the cash.

s69
00:07:47.880 --> 00:07:50.560
And so, that's exactly the same thing.

s70
00:07:50.560 --> 00:08:01.520
Our industry effectively misses this layer, this financial infrastructure that allows for this type of behavior to happen naturally, right?

s71
00:08:01.600 --> 00:08:04.880
And so, obviously this sounds like a great idea, though, right?

s72
00:08:04.880 --> 00:08:09.200
Like you're you're pitching this infrastructure, sounds like you have something to do with this.

s73
00:08:09.200 --> 00:08:11.640
Um But guys, this is not just me.

s74
00:08:11.640 --> 00:08:15.680
Uh in February, OpenAI, they have a quite a great team.

s75
00:08:15.680 --> 00:08:17.840
They're team called financial engineering.

s76
00:08:17.840 --> 00:08:27.360
And these guys basically published how they think about the architecture and what they had to build for this to work at the scale of OpenAI.

s77
00:08:27.360 --> 00:08:35.280
And what they had to say reads very similarly to how transactional systems of banks work.

s78
00:08:35.280 --> 00:08:43.719
Basically, you have a decision waterfall that needs to consider in runtime what is the client, the user, the agent is allowed to do.

s79
00:08:43.719 --> 00:08:49.920
And it needs to take into account many different conditions and policies of what is really allowed, right?

s80
00:08:49.920 --> 00:08:55.120
So, are you allowed or you have access to a certain feature or product?

s81
00:08:55.120 --> 00:08:58.800
What are the certain rate limitations under your plan?

s82
00:08:58.800 --> 00:09:02.560
Whether you are in a trial or in a certain promotional program.

s83
00:09:02.560 --> 00:09:07.000
Like you need all these decisions to be calculated immediately at front time.

s84
00:09:07.000 --> 00:09:10.720
And then it ultimately decide, can I give this user access?

s85
00:09:10.720 --> 00:09:11.480
Can they draw down?

s86
00:09:11.480 --> 00:09:12.720
Can they consume or not?

s87
00:09:12.720 --> 00:09:16.640
And all those decisions really can happen after the invoice.

s88
00:09:16.640 --> 00:09:21.640
They can't because you need to actually be able to calculate and prioritize this.

s89
00:09:21.640 --> 00:09:24.160
So, I think we have three things to notice from here, right?

s90
00:09:24.160 --> 00:09:28.520
The first thing is this is a single synchronous evaluation.

s91
00:09:28.520 --> 00:09:31.040
It happens at the request.

s92
00:09:31.040 --> 00:09:33.240
It doesn't happen after effect.

s93
00:09:33.240 --> 00:09:36.720
I think the second thing is the priority is deterministic.

s94
00:09:36.720 --> 00:09:43.240
So, we know in advance what are all the different rules that apply for certain things to be consumed, right?

s95
00:09:43.240 --> 00:09:47.800
And I think lastly, and here you guys, this is my opinionated view, you can disagree.

s96
00:09:47.800 --> 00:09:49.440
I don't think this is billing.

s97
00:09:49.440 --> 00:09:56.320
I don't think this is necessarily this entire architecture is just related to the idea of invoicing and billing.

s98
00:09:56.320 --> 00:10:02.280
This is a financial system that actually make decisions ahead of any invoice, ahead of any billing element.

s99
00:10:02.280 --> 00:10:10.920
And I think we're going to see more and more of this becoming a standard for other products and other AI systems as well.

s100
00:10:10.920 --> 00:10:16.920
Really just nerding just a little bit on I'm sorry, I just skipped one slide.

s101
00:10:16.920 --> 00:10:27.600
So, yeah, just just nerding just a little bit about what other trades or or elements we see and are used to in banks and we take them for granted,

s102
00:10:27.600 --> 00:10:31.200
but in AI they're not that granted, in software they're not that granted.

s103
00:10:31.200 --> 00:10:32.640
So, just one example, right?

s104
00:10:32.640 --> 00:10:42.680
Like if two separate parties have access to the same banking account, and that banking account has like $10, and they are entitled to draw down those $10.

s105
00:10:42.680 --> 00:10:46.839
What happens and when they both of them try to draw down at same time, right?

s106
00:10:46.839 --> 00:10:51.880
So, banking systems has the idea of holding, of settling.

s107
00:10:51.880 --> 00:10:58.640
You can really the concurrency is effectively solved with banks, but with AI and agents, it's not that case.

s108
00:10:58.640 --> 00:11:12.400
You could have whatever, like 10,000 agents trying to reach out to the same dollar pool, and they're going to try to do that all at once, and superficially it might look like they should be able to draw down because there is balance in the pool.

s109
00:11:12.400 --> 00:11:17.800
But, what happens when they all draw down from the same pool simultaneously, concurrency become an issue.

s110
00:11:17.800 --> 00:11:20.000
So, how do you solve for that?

s111
00:11:20.000 --> 00:11:26.200
This entire idea in financial systems actually called like it's a classic double double spend problem.

s112
00:11:26.200 --> 00:11:28.080
It comes from the accounting world.

s113
00:11:28.080 --> 00:11:43.200
It's a double entry bookkeeping, and as I'm sure you all are building shipping today with AI as the revenue scales, as your company scale, the idea of double entry accounting, double entry billing, the idea that you can solve concurrency,

s114
00:11:43.200 --> 00:11:49.080
these things would matter as you scale and they're not so easily solved down the line.

s115
00:11:49.080 --> 00:12:09.240
So, think about things like you know, can you basically introduce rates like hold and settle, think about idempotency and auditability of those uh requests and etc. Another thing that I think is very very

s116
00:12:09.240 --> 00:12:19.240
known in in you know, how banking systems works but not not necessarily enough established in AI is the idea that all the different pools have different sources, right?

s117
00:12:19.240 --> 00:12:25.640
So, in bank you have your debit account, you have your cash account, you have your savings, right?

s118
00:12:25.640 --> 00:12:32.280
And if you draw down, the bank actually knows how to calculate where you drawing down from and what does it mean?

s119
00:12:32.280 --> 00:12:40.560
Here, you know, say I have a pool of you know, one you know, 1,700 credits but they're not the same, they're not alike.

s120
00:12:40.560 --> 00:12:43.520
Each one of them is sourced from a different source.

s121
00:12:43.520 --> 00:12:47.320
So, how do you decide which which one of them do you draw down against?

s122
00:12:47.320 --> 00:12:51.200
Do you start with the ones that were you know, given granted earlier?

s123
00:12:51.200 --> 00:12:53.520
Do you start with the ones that were promotional?

s124
00:12:53.520 --> 00:12:57.600
So, how do you really calculate what's getting drawn down to the users?

s125
00:12:57.600 --> 00:12:59.480
I think that's that's a real problem.

s126
00:12:59.480 --> 00:13:08.120
Most companies today try to solve it like a single integer databases and that's not really going to work as you think forward.

s127
00:13:08.120 --> 00:13:10.080
Cool.

s128
00:13:10.080 --> 00:13:19.200
Um so really tying the knot the knot here, I think if you think about how we interact with software, it used to be flat.

s129
00:13:19.200 --> 00:13:25.920
So, it used to be just users and organizations and there wasn't a lot of meat in between them, right?

s130
00:13:25.920 --> 00:13:29.600
The the the hierarchy of you consume software was quite flat.

s131
00:13:29.600 --> 00:13:32.640
It was quite straightforward in terms of who see the value.

s132
00:13:32.640 --> 00:13:36.400
But today, I think effectively credit pools are no longer flat.

s133
00:13:36.400 --> 00:13:47.160
Uh you have a lot of different hierarchies in between who consumes different credits, and also they demand different governance, different logics, allocations, budgets, spend caps.

s134
00:13:47.160 --> 00:13:56.920
And effectively, what happens is that budgets are set at the top, where for instance, if you're selling to the enterprise, you're maybe selling like pre-commit annual contract.

s135
00:13:56.920 --> 00:14:05.680
Uh C-suite or somebody who approved that seven-digits contract, they expect to see how that credit are getting drawdown along the contract life.

s136
00:14:05.680 --> 00:14:08.200
But they don't just want to see that they were used.

s137
00:14:08.200 --> 00:14:17.200
They want to know who actually used them, which team, which users, uh which agents, uh to what extent, and they want to have fine-grain control over that.

s138
00:14:17.200 --> 00:14:27.720
That's literally the bare bones of how more and more companies thinking about selling AI workloads and how they they effectively work in production.

s139
00:14:27.720 --> 00:14:29.920
I'm going to run through some of this.

s140
00:14:29.920 --> 00:14:41.600
Um I think this idea of like what does it mean like high cardinality graphs and dimensions, being able to slice and dice against model types, users, like all this type of uh dimensionality

s141
00:14:41.600 --> 00:14:46.000
of uh of uh consumption is is probably interesting to to talk about.

s142
00:14:46.000 --> 00:14:50.520
But what I actually want to expand on is ultimately those four patterns.

s143
00:14:50.520 --> 00:15:01.800
So as you build AI and start selling AI at scale, you're effectively like four patterns that keep repeating themself, and they are very very much alike and and very similar to the idea of, you know, banking systems.

s144
00:15:01.800 --> 00:15:09.440
I think the first one we talked about earlier is basically being able to reserve before inference and settle actuals after.

s145
00:15:09.440 --> 00:15:19.480
Um this is like going to to be like a pattern that again and again we're seeing more and more often, and it has ramifications on how you do accounting, how you do

s146
00:15:19.480 --> 00:15:23.560
uh basically handle all those uh different concurrencies workloads and what have you.

s147
00:15:23.560 --> 00:15:32.080
I think the second thing is we're seeing more and more AI companies unexcited about the idea of sending all this usage data to to somewhere else over the cloud.

s148
00:15:32.080 --> 00:15:43.160
For latency reasons, for cost reasons, for data sovereignty reasons, there's a lot of good reasons for why these companies actually rather to keep the events data inside their VPC,

s149
00:15:43.160 --> 00:15:54.160
and I think we're going to see more and more AI companies seriously considering deploying this this architecture, this ledger, this metering within their VPC rather the the contrary of sending it over the internet.

s150
00:15:54.160 --> 00:16:09.840
I think for agentic checks, so think about how agents spawn, the idea of checking balances when agent does something you don't know in advance what they're ultimately going to do, how that how that task is going to basically cost.

s151
00:16:09.840 --> 00:16:16.080
So, if you don't know the cost, you need to be able to check those balances as the agent continues to get work.

s152
00:16:16.080 --> 00:16:23.520
And you need to be able to reserve an account for them in advance and then asynchronously after effect reconcile.

s153
00:16:25.680 --> 00:16:33.320
Quickly moving forward, so ultimately if you have those systems in place, what you really gain out of it is visibility.

s154
00:16:33.320 --> 00:16:42.080
So, it used to be given that for usage-based products you need visibility, but I think what happens today, you know, it goes beyond just plain visibility.

s155
00:16:42.080 --> 00:16:56.320
It it means that effectively like the ability to see for CFO, for CIO consumption of AI workloads across different models, across different features, across different products is not just, you know, a nice to have.

s156
00:16:56.320 --> 00:16:57.480
It's becoming a table stakes.

s157
00:16:57.480 --> 00:17:07.439
It becomes something that you can't do business without, and we're going to see more and more companies expect to have this off the bat before they actually do business with you.

s158
00:17:07.680 --> 00:17:18.920
And so really just giving some examples to how some of the best frontier AI companies are approaching some of this, I think you're you're seeing the patterns already there.

s159
00:17:18.920 --> 00:17:28.079
So, I think a lot of these companies already appreciate that you're no longer just selling software in the you know, commodity way that we used to think about subscriptions

s160
00:17:28.079 --> 00:17:29.000
and usage.

s161
00:17:29.000 --> 00:17:38.560
It's becoming more and more of a financial systems and financial systems require some of these complex ideas that weren't we're not there before AI.

s162
00:17:38.560 --> 00:17:47.360
Um and I think just to give like one small example that I talked about earlier, but I think might have gone a little bit uh hidden.

s163
00:17:47.360 --> 00:17:53.520
Um One of the changes in April that OpenAI did is really they didn't change the pricing.

s164
00:17:53.520 --> 00:17:57.200
All they did was change the pace in which tokens get burned.

s165
00:17:57.200 --> 00:18:02.600
So, effectively you use the same model, we pay the same cost uh per unit, but we burn more tokens.

s166
00:18:02.600 --> 00:18:05.320
So, that effectively was a pricing change.

s167
00:18:05.320 --> 00:18:07.400
How do you execute such a change?

s168
00:18:07.400 --> 00:18:09.960
Um because it's not just financial change, it's an infrastructure change.

s169
00:18:09.960 --> 00:18:13.320
You're basically change the rate of how the software gets consumed.

s170
00:18:13.320 --> 00:18:22.600
So, there's a lot of complexity not in terms of just how we change pricing, but also how the infrastructure underneath allows for more complex ideas into how value is getting

s171
00:18:22.600 --> 00:18:24.960
uh uh introduced to the users.

s172
00:18:24.960 --> 00:18:33.600
And really just, you know, before wrapping up, you know, there our team likes to say, and this is something we started to say more and more often than before,

s173
00:18:33.600 --> 00:18:43.400
it really feels like you know, there was in the in the early 2010s there was this Stripe moment where every company wanted to ship product into the internet and you need a checkout

s174
00:18:43.400 --> 00:18:49.880
and you need it to be secure and it and the developers need to to to to to make it easy for them to quickly get them up and running.

s175
00:18:49.880 --> 00:18:59.840
I feel like where we are today is those ideas of like building a banks, it's okay if you feel like, I don't know, you like wearing suit and tie, maybe you want to build a bank,

s176
00:18:59.840 --> 00:19:11.280
but if you are a building a bank, you probably want to be aware to all of these traits and elements that has to be there so you don't meet them down the line where where it's get really, really more complex to fix in a hindsight.

s177
00:19:11.280 --> 00:19:19.560
And so, feels like the industry really misses its Stripe moment where all these constructs just exist and it's they're so easy to use.

s178
00:19:19.560 --> 00:19:28.000
And you know, there's a lot of excitement from our team when it comes to the potential of having such systems ready and help AI companies keep shipping.

s179
00:19:28.000 --> 00:19:32.400
Um if any of that was interesting, really come by our booth.

s180
00:19:32.400 --> 00:19:37.120
We are in a quite of a weird location right at the start, but still there.

s181
00:19:37.120 --> 00:19:38.400
I know there's a game right now.

s182
00:19:38.400 --> 00:19:39.040
Come visit.

s183
00:19:39.040 --> 00:19:40.320
We're demoing the product.

s184
00:19:40.320 --> 00:19:43.200
Feel free to ask any questions and thank you for coming today.

s185
00:19:43.200 --> 00:19:45.760
It was awesome to to have you have you with me.

s186
00:19:45.760 --> 00:19:50.400
And yeah, love to nerd some more by our booth.