Buying GPUs Should Work Like Buying Stocks — Carmen Li, Compute Exchange

AI Engineer · 18 min · 240 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:12Happy Wednesday.
00:13My name is Carmalee.
00:14I'm the CEO for Compute Change as well as Silicon Valley.
00:18Um I love the you know, the previous session.
00:22Um we talked the talk about a lot about the whole ecosystem, the financiers, and the small market traders, and the hedgers.
00:30So, what we're sitting in is really cool.
00:32I've company of Compute Change.
00:34The world's actually the only neutral marketplaces for GPU procurement.
00:40We started in 2024 when there were, I don't know, five new clouds, three hyper scalers.
00:46Um essentially, we sort of will foresee, interestingly enough, the proliferation of the GPU ecosystem.
00:55So, you think about stock market today, right?
00:57When you try to buy and sell stocks, you can call brokers.
01:01Obviously, you can call a few banks to say, "Hey, I want to buy and sell X amount of Tesla or actually SpaceX, right?"
01:08And you can do that or you can transact through your app at NYSE or NASDAQ.
01:14Doesn't mean you get the best price at NYSE or NASDAQ just because it's more efficient.
01:19And you know the liquidity is going to be there.
01:21You know the trade will be settled.
01:23It'll be cleared.
01:24You can get what you're looking for at the prices you're looking for in a very efficient manner, right?
01:29For capital markets and resource allocation, it's about efficiency.
01:34So, in computer world, obviously now we have a huge proliferation, which is great.
01:40We have over 300 new cloud private provider globally.
01:43We have a lot of token factories.
01:45We have a lot of API routers, which is all good for end customers.
01:50But at the same time, introduce a very interesting variable is how do we make transaction efficient, right?
01:58Do you want to call 300 sales people?
02:00I mean, it's great to make friends, right?
02:02You want to talk to them regular basis.
02:04They know a lot of things.
02:05But same time, when you need compute in the next 48 hours on reserved contracts, is easiest way is go to exchange and then go to the verified computer resources.
02:16Just like go to Nasdaq and I see to buy stocks.
02:19Right?
02:20So, this one of the things computing does extremely well is we have this neutral platform.
02:25And by design, we do not own GPUs.
02:29Once we start owning GPUs, that would translate to just like Nasdaq owning stocks, translate to I have a conflict of interest with you and then with the providers.
02:38I could cherry-pick clients, right?
02:41I could hoard relationships.
02:43Because Compute Change does not own anything by design, we work really well with all our providers globally.
02:51Um and one very important thing is we treat every single transaction individually, meaning we want to find you as an customer the most suited computer resources.
03:05And we focus on reserved compute forward contracts as well as token packages.
03:10So, give you some um high-level.
03:16So, today we have over 100 verified providers um and over 80,000 GPU in supply.
03:25We cover all regions, obviously ex-China.
03:28[gasps]
03:29Um our commitment is 24-hour turnaround.
03:33The difference between us and other so-called brokers is we do verify GPUs.
03:39If it's existing reserved capacity, meaning things are actually in a functional stage, we'll verify the UUIDs, the inventories, the performances.
03:49We run Nvidia DGX, Inference X, MLPerf, make sure you get resources you promised to get.
03:56Um so we're in the middle middle sort of trust the platform.
04:00So if it's forward capacity, I'd say you need GPUs starting Jan 1st, 2027.
04:06We actually look through the purchase orders, all the permits they get, make sure, you know, we help you do the due diligence level.
04:14And on the flip side, right?
04:15If you're a new cloud, you can send this this is a zillion clients looking for compute.
04:20How do you know who has the ability to pay at a scheduled rate?
04:25That's where Compute Exchange comes in.
04:26We do KYB KYC of the clients.
04:29Obviously, you want to make sure your clients is the clients you think they they are.
04:33So we step in and do that kind of verification just like any exchange would do.
04:38And everything we do in the forward markets is actually a precursor to physically delivered futures.
04:45Also give you some context why I am what I do.
04:47Sorry, I probably shouldn't say that.
04:49I start my career as energy option market maker um at DRW Trading, Chicago.
04:54So I did that for 4 5 years.
04:56So as option energy market maker, everything's very intuitive.
05:00You have WTI, you have Brent, right?
05:02WTI is a physically delivered commodity contracts.
05:06Brent is cash settled.
05:08They also have very distinct use cases.
05:11You're looking for hedging in the financial sense, right?
05:15The your price volatility, your cost volatility, you're going to Brent.
05:19You're looking physically procure resources, use WTI.
05:23So we're creating that the WTI at Compute Exchange.
05:30So the ways to buy compute through um Compute Exchange right now, you can get reserved contracts, you can get forward contracts on both GPU and token.
05:39Let me explain.
05:40The forward contracts on GPU is very reasonable, right?
05:43You want something starting in the future, nothing available right now.
05:46So that's something we're doing on regular basis.
05:49Token forward I think can be a newer-ish concept.
05:52Basically, you can think about as a dedicated deployment, on-premise deployment, right?
05:58Similarly, you reserve a bare metal GPU clusters.
06:02You reserve a cluster, but you have dedicated token deployment on top of that.
06:07So, you get guaranteed SLAs for that particular models.
06:11You get quantization, precision, everything's fixed for you for a period of time.
06:17So, we try to solve the on-demand token API volatility pricing problem here.
06:24The third is we have a huge market, open hardware market.
06:28It's really interesting.
06:29The secondary server transactions started last year when people just pinging me say, "Hey Carmen, I have like 20,000 H100 sitting at data center.
06:40Can you take away from me?"
06:41So, I was like, I don't know.
06:43I can't store it in my home, obviously.
06:45So, we started having a marketplace for secondary server sales.
06:49This has been a very vibrant market for the past I would say 12 months.
06:54We cover again all servers, even B200 you see secondary transaction is happening.
07:00Sometimes trading above MSRP, right?
07:03Can make sense think about like COVID times when secondary cars trading above, right?
07:08It's a brand new car price.
07:11So, we covered the three major layers of compute.
07:17So, reserve GPU, obviously, I think the previous speaker covered extremely well with the terms.
07:23The minimum we do is one node starting at one month, and then we can go all the way to three, four, five years, right?
07:29It's all depends on the availability, the providers, but you get a chance to state your preferences.
07:36We don't do on-demand.
07:40And another thing really works really well for config changes we do provide a flexibility.
07:46Let's say you have over committed it in terms of quantity or duration, you can sell it back to config change, right?
07:52So, that's something we committed to do.
07:54So, you don't have to worry too much if you over bought.
08:00All right, segment life So, I will actually do a quick demo later.
08:04Um so, we have a lot listings, marketplace listings you can browse and chat with the providers and figure out the best silver mechanisms.
08:12You can also RFQ if you don't see anything you like you liked.
08:15We actually seen over a billion RFQ'd every month.
08:21Sounds big, but we all know it's full computer it's not that huge, right?
08:25But still you see the the the the the activities, which a lot is we see a big range of RFQ requests from one node all the way to 20,000
08:36nodes.
08:37So, our clients are you know, all different shapes and forms, different use cases, which is great.
08:45Yeah, so you set timing submit and we route it.
08:47So, in the behind the scene, we have inventory tracking.
08:51So, with over 100 providers globally, we know who has one roughly one right before capacity.
08:57And then I say you you submit a request in that hey, I need 30 300 GPUs B200 US East starting Jan 1st.
09:05We say, you know, we have three providers we think they have those.
09:09We'll route them to the three.
09:10They will respond to you through the platforms and they'll give you um a rubrics back.
09:15Pricing, specs, SLAs, we benchmark them if it's existing servers and then it's your choice, right?
09:22Price is not only indicator.
09:24We believe give you as much transparency as we can and then you will make a decision.
09:32So, everything is in one place.
09:33Again, the rubrics come back.
09:34You will see everything and we introduce you to each other.
09:37We never hold relationships, right?
09:39We'll make sure you know who you're dealing with.
09:42Um so, buy tomorrow's computer today's price.
09:45So, it's a physically procured platform.
09:49This is the used refurbished uh GPUs.
09:53So, again, we have a extremely um exciting uh ecosystem participants in this uh in this uh secondary GPU cells.
10:02We cover from 490s to B1200, AMD, everything.
10:10Inventory is life and we do connect with you through all different channels like email, chat, however you want to do this.
10:18Um we track end to end.
10:20We actually have scoring for everybody.
10:23We score you, our customers.
10:25We score our providers.
10:27You can't see it quite yet, but you know, it'll be really fun.
10:30Um we're going to have end of year celebration for our providers.
10:33We can We're going to give all the best provider in APAC, best provider in Europe, best provider in US, most responsible providers.
10:42So, we're going to have a lot of a provider awards coming up.
10:44Stay tuned.
10:48Um verify network.
10:49So, everything's uh you know, verified, which is you know, negotiate state exchange.
10:54So, we help in terms of be the middleman.
10:57Sometimes things are not very clear.
10:59We try to make sure people understand what's the best practices we have seen in the ecosystem.
11:07I'm going to quickly show you guys the platform.
11:11I think that's more fun.
11:19Uh here.
11:22So, this is the platform.
11:24You can see a bunch of available inventories.
11:26By the way, anyone can come sign up.
11:28It's completely free.
11:28There's no downside using us.
11:30You don't have to buy, you can browse.
11:33You can request for quote if you, you know, interested something we don't have.
11:37You can start with your vendor type.
11:39Again, we agnostic of GPUs.
11:42You can select your model, you can select your deployment type.
11:45You can select number of GPUs.
11:48You can select your compute interconnect preferences, CPU platforms, storages, everything you need.
11:55And then some optional items with system software stacks, regions.
11:59Obviously, we encourage you to select multiple regions if you open to that.
12:02You can add head nodes, tell us your use cases, anything you want, and then submit RFQ.
12:09So, this has been extremely robust.
12:11And then we, you know, we obviously we want to make sure um if you have any feedback, let us know.
12:18So, I'm going to pause on Compute Change side.
12:20Um I think I have some time left, which I'd love to show you some of numbers we've seen in terms of term structure.
12:29Also, context.
12:31I'm CEO for Compute Change.
12:32I'm also CEO for another company called Silicon Data.
12:35So, I'm a founder CEO for Silicon Data, which is the Bloomberg for compute.
12:40So, we provide indices for CME to settle their futures on, which will be happening in a few weeks.
12:47We I'm very excited about this.
12:49So, the one that's going to be start trading at CME will be the H100 near cloud on demand rate as well as V200 um prices.
13:00What I want to show you is implied forward curve.
13:03So, as I said, Compute Change we do a lot of forward capacities.
13:07So, essentially a lot of longer term contracts is going to be very critical using about even in your cloud or your your consumers.
13:16Let me just make it larger.
13:18So, the yellow line was last year, right?
13:21The the X horizontal line is the duration, and the dollar amount is the Y axis.
13:28So, last year was complete downward sloping curve for the yellow line, right?
13:32Meaning the longer term contract you signed, the cheaper you know, was per shift per hour.
13:38And this year, starting early this year, the whole curve shift up and became contango.
13:43If you see the the red line, right?
13:45So, translate to people not getting a discount enough discount for longer term contract.
13:52Implies that people assume the price going to rise down the line.
13:56And now you see the price the whole curve keeps shifting up, got flattened, right?
14:01So, meaning people are now either get penalized or get, you know, awarded by signing long term contract.
14:08This is reserved contract we observe.
14:10And if you look at implied forward rates, this is not this is the implied forward rates, right?
14:15Because we don't have a future trading.
14:17So, this is calculated based on our term structure.
14:21And we use similar methodology as Treasury futures curve.
14:24So, you can see a market consensus view is the price could go up next in the the future next 36 months.
14:32Again, market consensus doesn't mean it's right.
14:36If you look at B200, again, last year was contango and this year the whole curve shift up.
14:42Every single term is more expensive and it's got flattened, right?
14:47Implied forward rates, similarly, people think the price going to go up.
14:51I want to point this out the term structure how is this shows you that GPU is behaves just like any commodity, have a very clear term structure risks, which data futures are excellent at hedging.
15:05So, we have seen a lot block trading happening around the term structure.
15:10So, you think about this, right?
15:11If you have servers on your balance sheet, you have financing obligation for next 5-6 years, and you have reserved contracts for first three, then you're exposed essentially year four, five, or six, right?
15:24So, you want to hedge that risk out by entering, you know, a swaps or block trading.
15:30Um so, those are things Oh, or there there are trades happening even for the first three years, which you have resale contracts on.
15:38What if the resale contract defaulted, right?
15:41What's your um guarantee from some of the players?
15:45So, we saw block trading happening around the indices.
15:48Trade as a floating rate.
15:54This chart um actually is being uh used differently by the NVIDIA team.
16:02Um in NVIDIA's latest uh um September investor day deck, they said us with only data provider that actually cited in our deck.
16:12Um this is very interesting.
16:14Silicon data becomes the appraisal for a few banks when they have GPU servers on their balance sheet.
16:20Um this everything you see here uh forward-looking view, right?
16:24So, it's discounted cash flow.
16:27So, as I said, we have a time structure.
16:29So, assume A100s, right?
16:31A100s have been around for six years.
16:33If today you sign a three-year long lease on A100s, and you have a rate, and we discount the the cash back to time zero, you have how much the asset's worth,
16:44right?
16:45Obviously, there's a lot of things baked in, utilization rate, what is your cost.
16:50There's a bunch of assumptions which we're pretty transparent about.
16:54Um we actually calculate the residual value for normalized value, obviously, across all A100, H100, B200 servers globally.
17:03And then we'll compare that with the straight-line depreciation, which is everybody using the five-year depreciation.
17:09What's interesting is the A100, right?
17:11Even though it is downward sloping, you know, think means things are a but it's worth the accounting depreciation curve, straight line.
17:20And the H100 obviously, you know, it's going going bananas now and the price actually went up.
17:26My My analogy would be just like secondary cars during COVID time.
17:31The price went up, right?
17:33And B200 obviously went above its MSRP by this point.
17:38So, this is give some give you some views about what's going on in the secondary market side.
17:42I think I'm out of time, but you know, feel free to follow me at Carmen CLI at computerhistory.com, CLI@computerhistory.com.
17:51Feel free to get me on Twitter.
17:52Carmen Li is my Twitter handle.
17:54Just Carmen Li.
17:56That's it.
17:57Thank you.