WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=CoEIs6Xm8m8

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/CoEIs6Xm8m8.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.800 --> 00:00:18.800
Hi, I am SA um founder of Klein.

s3
00:00:18.800 --> 00:00:23.760
Uh I started Klein as an open source project a few years ago.

s4
00:00:23.760 --> 00:00:35.440
Um, some of you might know it as the first ever coding agent back before um the cloud Mac subscription and the codec subscriptions when people had to pay for each and every API request

s5
00:00:35.440 --> 00:00:37.840
which uh got extremely expensive.

s6
00:00:37.840 --> 00:00:50.160
Um this was before prompt caching became a thing and so there were people that paid hundreds of dollars a day uh using client but for a lot of people it was their first AGI moment.

s7
00:00:50.160 --> 00:00:55.120
It was the first time they saw um LLMs be able to do their jobs end to end.

s8
00:00:55.120 --> 00:00:56.879
Um and they got hooked.

s9
00:00:56.879 --> 00:01:02.239
Um and I don't think Klein would have been as successful as it as it is if it wasn't open source.

s10
00:01:02.239 --> 00:01:12.400
Um because it allowed these developers to inspect our code and trust it and uh connect to any API so they could be comfortable with spending so much money on it.

s11
00:01:12.400 --> 00:01:17.520
Um and uh know that they weren't getting screwed over.

s12
00:01:17.520 --> 00:01:22.640
Um, and we were the first to add things like custom rules and and plan mode.

s13
00:01:22.640 --> 00:01:28.720
And a lot of that came from talking to and learning from this really incredible open source community we had around the project.

s14
00:01:28.720 --> 00:01:41.439
Um, and so, you know, having spent most of my life building open source, it's really heartbreaking to see just like the broader open source community uh wither and and die over the last two years

s15
00:01:41.439 --> 00:01:45.680
uh because of how AI has fundamentally changed everything about software development.

s16
00:01:45.680 --> 00:02:00.000
Um, GitHub is effectively uh an archive of SLOP PRs and issues and security reports um where the sense of community before has turned into this like deep skepticism and distrust

s17
00:02:00.000 --> 00:02:03.200
um of each other's responsible use of these tools.

s18
00:02:03.200 --> 00:02:13.040
Um because AI coding can be extremely dangerous to a project and everyone's kind of had to learn that on the fly, but especially open source projects uh that rely on

s19
00:02:13.040 --> 00:02:15.120
trusting third parties.

s20
00:02:15.120 --> 00:02:21.840
Um, and so I wanted to share some examples of how open source has been dealing with AI.

s21
00:02:22.000 --> 00:02:28.400
So this is the code of conduct for Zigg, which is the language that powers bun.

s22
00:02:28.400 --> 00:02:32.080
Um, and they essentially ban all use of AI.

s23
00:02:32.080 --> 00:02:36.400
You can't use it on pull requests or issues or even comments.

s24
00:02:36.400 --> 00:02:47.360
Um, and the reason for this is that to them, the core Zigg team, um, they value contributors more than they do the contributions.

s25
00:02:47.360 --> 00:02:55.440
Um, and so the primary goal for reviewing PRs and things isn't to add new code, but it's to help grow new contributors who can become trusted um, over time.

s26
00:02:55.440 --> 00:02:58.080
And AI assistance completely breaks that.

s27
00:02:58.080 --> 00:03:12.319
Um, this is an uh a post from the CEO of Kurl who says that his project is effectively being dodoed by AI generated bug reports and they're even considering

s28
00:03:12.319 --> 00:03:17.120
um shutting down their bug bounty program for the first time in decades.

s29
00:03:17.680 --> 00:03:21.200
Um, and this is TL Draw.

s30
00:03:21.200 --> 00:03:27.440
Uh, they're automatically just closing all poll requests whether they're AI generated or not.

s31
00:03:27.440 --> 00:03:35.280
Um, and it's gone so bad that GitHub added a feature to disable thirdparty pull requests altogether.

s32
00:03:35.280 --> 00:03:40.080
Um, which is which is really sad because pull requests were the thing that made GitHub what it is today.

s33
00:03:40.080 --> 00:03:45.599
And we're probably going to see a lot of big open source projects um, opt into this.

s34
00:03:45.599 --> 00:03:56.799
Um, and so when I say that open source is dead, I mean some parts of it like the community um, it's just not worth cultivating anymore, especially because building software is so cheap.

s35
00:03:56.799 --> 00:04:01.519
Um, and uh, also the risk of supply chain attacks.

s36
00:04:01.519 --> 00:04:05.200
I'm sure you've seen all the all the reports of of things getting compromised.

s37
00:04:05.200 --> 00:04:14.720
It's become more dangerous than ever to depend on third party software uh, where it takes a single compromise and a massive chain of contributors to get pawned.

s38
00:04:14.720 --> 00:04:20.639
Um, so just as an example, light LLM um, is a Python package.

s39
00:04:20.639 --> 00:04:23.120
It gets like three and a half million downloads a day.

s40
00:04:23.120 --> 00:04:37.199
They were compromised for three hours where attackers um used a GitHub app that they used um to steal their Pippi publishing tokens and um publish a compromised version of the package

s41
00:04:37.199 --> 00:04:49.360
that would install a credential harvester that would steal your API keys, your SSH keys, your crypto keys, um and also install a backdoor that lets them do remote command execution.

s42
00:04:49.360 --> 00:05:02.800
And the only reason this was even caught as quickly as it was was just pure luck um because the malware had a bug in it where it would cause cursor to crash if you ran the light LLM MCP server.

s43
00:05:02.800 --> 00:05:07.759
And a security researcher noticed that um and was able to figure it out.

s44
00:05:07.759 --> 00:05:16.000
But if this had been out any longer, it would have caused like catastrophic damage, especially because a lot of the people using Light LLM are like the enterprise customers

s45
00:05:16.000 --> 00:05:20.560
um and developers um that have their own internal gateways.

s46
00:05:22.560 --> 00:05:29.680
Um but despite all of this, I believe there are some parts of open source that are sticking around and becoming more important.

s47
00:05:29.680 --> 00:05:39.520
um like allowing others to use your thing and uh freely in the public domain and build on top of and those parts about it are going to become more important than ever

s48
00:05:39.520 --> 00:05:48.479
um particularly with open weights models because of the economic impact and so to help explain why I want to look at what's happening with inference spend right now.

s49
00:05:48.479 --> 00:06:05.759
So, this is a report from uh an anonymous report from a CFO at an unnamed company where they accidentally spent $500 million on Claude in a single month because they didn't set the usage limits on their thousands of employees on their anthropic dashboard.

s50
00:06:05.759 --> 00:06:14.400
Um this is another report by Uber CTO where after they rolled Cloud out to their organization, uh 95% of their engineers were using it.

s51
00:06:14.400 --> 00:06:24.639
75 70 70 70% of their committed code came from claude and their monthly spend per user was up to $2,000 and they said they used their entire 2026 budget

s52
00:06:24.639 --> 00:06:31.440
um in just four months and the crazy part is is that the AI labs are losing money too.

s53
00:06:31.440 --> 00:06:42.960
Um this is a chart from semi analysis where they ran experiments with claude code and codec subscriptions where they would give them long u horizon coding tasks until they exhausted their weekly limits.

s54
00:06:42.960 --> 00:06:49.919
And they found that a $200 plan for claude would give them about $8,000 worth of API usage.

s55
00:06:49.919 --> 00:06:56.639
Um and a $200 subscription to codeex would give them about $14,000 worth of API usage.

s56
00:06:56.639 --> 00:06:58.800
Um so I think the strategy is pretty obvious.

s57
00:06:58.800 --> 00:07:13.199
They're essentially going to subsidize this um until they have as many engineers dependent on their uh tooling as possible with agents in their CI and background cloud agents and looping agents and all these things where it feels like

s58
00:07:13.199 --> 00:07:21.360
every new feature and marketing push from these labs seem to be a new workflow to standardize on to use even more tokens and to be locked in even more.

s59
00:07:21.360 --> 00:07:29.759
Um, and then inevitably the price gouging, uh, once they've got you trapped where your developers can't work without the tools.

s60
00:07:29.759 --> 00:07:31.840
And this isn't theoretical.

s61
00:07:31.840 --> 00:07:35.840
We're seeing this happen live with some of the customers that we talk to.

s62
00:07:35.840 --> 00:07:46.080
Um, so just a quick show of hands, how many of you basically stop working whenever there's a cloud outage or a GPT outage?

s63
00:07:46.080 --> 00:07:47.675
Yeah, same.

s64
00:07:47.675 --> 00:07:48.400
[laughter]

s65
00:07:48.400 --> 00:07:57.599
Um, I think that's a reason why we've seen Anthropic and OpenAI go from being API businesses to investing so much into the application layer.

s66
00:07:57.599 --> 00:08:06.160
Um, is because they know that that's where they can set these sorts of traps and build their moat for the day that these models inevitably become a commodity.

s67
00:08:06.160 --> 00:08:08.720
But I don't actually think the strategy is going to work.

s68
00:08:08.720 --> 00:08:12.160
Um, and that's the message I wanted to get across today.

s69
00:08:12.160 --> 00:08:14.479
Uh, that this feels very shortsighted.

s70
00:08:14.479 --> 00:08:26.560
And what we're noticing happen um in the world is that it doesn't matter how many features your CLI agent has, uh developers and businesses will just jump to whatever offers them the best value for their dollars.

s71
00:08:26.560 --> 00:08:39.599
So if we look at current open weights models, many of which are are built in China, we'll notice that although they've lagged behind the American closed source competitors, we're at an inflection point where raw intelligence lead doesn't matter as much anymore.

s72
00:08:39.599 --> 00:08:43.919
Um because these models are powerful enough where you don't always need the best one for all your work.

s73
00:08:43.919 --> 00:08:50.160
And that cost is becoming extremely important to these businesses that have kind of turned a blind eye until now.

s74
00:08:50.240 --> 00:09:00.640
And I think we all kind of feel it that to get the best output from these models, it's more a problem of what context and tools you give the agent access to and less about its raw intelligence.

s75
00:09:00.640 --> 00:09:16.160
with the right AI native development infrastructure with project skills and rules um systems of verification and quality gates um even a mediocre model can produce similar results as a more intelligent model it just might take more tokens

s76
00:09:16.160 --> 00:09:27.680
um the intelligence is better placed in the system and guard rails around the model so that you don't have to be as reliant on the model or your end developer's responsible use of the model itself.

s77
00:09:28.240 --> 00:09:36.800
So we recently shared uh an anecdotal experience where we were skeptical of the benchmark saying that GLM was better than Opus.

s78
00:09:36.800 --> 00:09:45.680
So we tested them on a real bug from the client repo and while both models fixed the issue, GLM was the winner in terms of cost and code quality.

s79
00:09:45.680 --> 00:09:50.000
So GLM used twice as many tokens but only cost half as much.

s80
00:09:50.000 --> 00:09:51.760
Opus finished faster.

s81
00:09:51.760 --> 00:09:54.480
It used half as many tool calls.

s82
00:09:54.480 --> 00:10:01.440
But GLM cleaned up dead code and verified that the build compiled before completing while Opus didn't.

s83
00:10:01.440 --> 00:10:04.640
It left a bunch of type errors and it broke the production build.

s84
00:10:04.640 --> 00:10:11.440
Um, and so that gave us the sense that GLM was trained to spend more tokens verifying its output.

s85
00:10:11.440 --> 00:10:18.000
Um, which is fine because the tokens are cheaper any anyways and it's really the end result that matters.

s86
00:10:18.000 --> 00:10:26.959
And because these open weights models can deliver the same output, a little bit more tokens, we're seeing signs of the industry adopting and standardizing on these models.

s87
00:10:26.959 --> 00:10:40.160
So this is Brian Armstrong, the CEO of of Coinbase, saying that they've uh defaulted to using GLM and Kimmy in their internal LLM gateway and that this has cut their AI spend by nearly half

s88
00:10:40.160 --> 00:10:42.959
uh while their token usage continues to grow.

s89
00:10:42.959 --> 00:10:50.079
And I think we'll see other businesses building their own internal tooling and routing to work with these agents in the most dollar efficient way for them.

s90
00:10:50.079 --> 00:10:56.480
Even if it means not having access to the latest new feature in something like cloud code.

s91
00:10:57.519 --> 00:11:02.240
We're seeing the same thing that happened with open compute 15 years ago.

s92
00:11:02.240 --> 00:11:06.320
So uh just a quick history lesson for those that haven't heard of this.

s93
00:11:06.320 --> 00:11:13.519
In 2011, Facebook was just getting started on building out um stuff like distributed computing infrastructure and data centers.

s94
00:11:13.519 --> 00:11:17.200
Uh but by the time they built it, Amazon and Google already beat them to the punch.

s95
00:11:17.200 --> 00:11:19.360
So it wasn't a competitive advantage.

s96
00:11:19.360 --> 00:11:23.200
So Mark said, "All right, let's just open source it and see what happens."

s97
00:11:23.200 --> 00:11:31.680
Uh so they took the designs for their data centers and their servers and their networking and cooling racks and all the physical hardware they spent all this energy and and money

s98
00:11:31.680 --> 00:11:33.760
um building and they just gave it away.

s99
00:11:33.760 --> 00:11:39.600
They published uh schematics and CAD files and everything uh and called it the open compute project.

s100
00:11:39.600 --> 00:11:44.480
And what they saw was that the entire supply chain reorganized around it.

s101
00:11:44.480 --> 00:11:48.720
So before open compute, every company designed its own proprietary servers.

s102
00:11:48.720 --> 00:11:54.800
So manufacturers were doing small production runs um of like very custom hardware which was expensive.

s103
00:11:54.800 --> 00:12:06.560
But when Facebook's designs became this sort of shared open standard, suddenly everyone was ordering the same thing and manufacturers could do these massive standardized production runs commoditizing these components.

s104
00:12:06.560 --> 00:12:14.320
Um so no single vendor could charge a premium and the price of everything came down uh for the whole industry including Facebook itself.

s105
00:12:14.320 --> 00:12:23.040
Um and so what Facebook found was by giving these designs away they created the market that drove their own costs down and saved them billions of dollars down the road.

s106
00:12:23.040 --> 00:12:33.120
And so I think the lesson um taught here is that the industry will adopt and standardize on something that they can build on top of even if it isn't the best thing.

s107
00:12:34.079 --> 00:12:41.920
And with how much capex we've locked in for the next 5 years for AI infrastructure buildout, open weights models are only going to get cheaper.

s108
00:12:41.920 --> 00:12:50.800
Um there's estimates that we'll spend nearly three trillion dollars and create uh over a 100 gawatts of new data center capacity by 2030.

s109
00:12:50.800 --> 00:12:54.560
um roughly doubling uh global capacity today.

s110
00:12:54.560 --> 00:13:01.360
And hosting providers like base 10 and fireworks, their whole purpose is to beat each other u to beat the competition.

s111
00:13:01.360 --> 00:13:10.800
So they'll use infrastructure efficiency gains like dedicated hardware and caching and batching volume tricks and inference specialized silicon to drive cost down even more.

s112
00:13:10.800 --> 00:13:20.160
And um by 2030, the estimates are that inference on a 1 trillion parameter LLM will cost um 90% less than it does today.

s113
00:13:20.880 --> 00:13:28.399
we're seeing uh the same sort of cost cutting tricks that commoditized the cloud 10 years ago um happening in inference.

s114
00:13:28.399 --> 00:13:38.959
Uh so in 2014 uh Google uh at their GCP live March announcement said that they would cut compute by 32% and storage by 68%.

s115
00:13:38.959 --> 00:13:47.120
Um and AWS fired back with similar cuts within within days and this was AWS's 42nd uh price cut at that point.

s116
00:13:47.120 --> 00:13:58.800
And so from 2015 onwards once raw compute and storage were a commodity the hyperscalers stopped competing on on it as much and started competing on other things like databases and serverless.

s117
00:13:58.800 --> 00:14:13.839
And I think because open weights allows these host providers to compete and cost uh optimize so aggressively, we'll see mass adoption of these foreign open weights models um because when dollars are involved, the markets are extremely efficient

s118
00:14:13.839 --> 00:14:21.040
and the absurd API costs that these closed labs charge just won't be worth it anymore for most knowledge work.

s119
00:14:22.480 --> 00:14:30.160
Uh and so this is me just like humbly requesting the American labs to take open weights more seriously.

s120
00:14:30.160 --> 00:14:39.440
Uh because before we know it, all this infrastructure that we're investing in could be built on foreign models that take the world by a storm and make GPT and cla irrelevant.

s121
00:14:39.440 --> 00:14:51.040
um mind share and adoption is incredibly important and there's a chance that if the foreign models become the standard there won't be a reason to switch back to GPT or claude or Gemini no matter what the marginal improvements are

s122
00:14:51.040 --> 00:14:54.800
and then we lose control over the development of this technology.

s123
00:14:54.800 --> 00:15:05.920
Um and who knows where the world is is headed if we don't have the likes of anthropic and open AI to invest so heavily into safety research in ways that perhaps these other labs wouldn't.

s124
00:15:05.920 --> 00:15:19.199
I think the development of a technology this transformative is deeply uh tied to the ideals of the people and the nation that's building it and to instill those values in the future of this technology we need to keep the lead.

s125
00:15:19.199 --> 00:15:22.240
So I don't mean we need to open source our research.

s126
00:15:22.240 --> 00:15:23.839
I think that's what gives us the lead.

s127
00:15:23.839 --> 00:15:24.215
I think

s128
00:15:24.215 --> 00:15:24.240
[snorts]

s129
00:15:24.240 --> 00:15:33.519
we need to open we need to open up our models and uh start releasing more open weights models which as we know are not nearly as useful.

s130
00:15:33.519 --> 00:15:45.040
you can use um and extract the traces and and train more you know your copycat models on them more easily but not in a way that can leaprog and

s131
00:15:45.040 --> 00:15:59.519
this would make models uh more usable by the industry in a way that allows more competition and adoption and better price and value for customers and I think that's how we keep our lead during this very critical moment

s132
00:16:01.040 --> 00:16:11.600
me and client believe so much in this open weights future that we launched uh an openw weightight subscription plan earlier this week that through volume uh based discounts and partnerships

s133
00:16:11.600 --> 00:16:23.920
uh with inference host providers we can offer significant discounts compared to paying for these models um at direct API cost um and we plan on continuing to increase the usage quota for these models as they become cheaper

s134
00:16:23.920 --> 00:16:27.440
uh you can sign up at that link client.bot/pass bot/pass.

s135
00:16:27.440 --> 00:16:38.160
Uh if you'd like to get a feel for how far models like GLM and DeepSeek have come and how you don't need the most expensive closed frontier model access to get work done anymore.

s136
00:16:38.160 --> 00:16:43.920
Uh client is also uh open source and you can bring an API key and use any other provider.

s137
00:16:43.920 --> 00:16:47.600
Um you can use it on your CLI and VS code and Jet Brains.

s138
00:16:47.600 --> 00:16:55.120
Um and uh yeah, we continue to um add the newest models whenever they're released.

s139
00:16:55.120 --> 00:17:06.400
So it's a good way to get a feel of like how much better the latest newest model is especially with open weights because um you can't access those with like the cloud or chat GPT subscriptions.

s140
00:17:07.280 --> 00:17:11.120
Uh cool that is my presentation.

s141
00:17:11.120 --> 00:17:12.291
Thank you.

s142
00:17:12.291 --> 00:17:14.291
[applause]

s143
00:17:26.702 --> 00:17:28.702
[music]