Buying GPUs Should Work Like Buying Stocks — Carmen Li, Compute Exchange
https://www.youtube.com/watch?v=laQ8wMysnkk
AI Engineer · 18 min · 240 sentences · sentence-level transcript from YouTube's caption track; each line begins at the time shown

[00:01] [music]
[00:12] Happy Wednesday.
[00:13] My name is Carmalee.
[00:14] I'm the CEO for Compute Change as well as Silicon Valley.
[00:18] Um I love the you know, the previous session.
[00:22] Um we talked the talk about a lot about the whole ecosystem, the financiers, and the small market traders, and the hedgers.
[00:30] So, what we're sitting in is really cool.
[00:32] I've company of Compute Change.
[00:34] The world's actually the only neutral marketplaces for GPU procurement.
[00:40] We started in 2024 when there were, I don't know, five new clouds, three hyper scalers.
[00:46] Um essentially, we sort of will foresee, interestingly enough, the proliferation of the GPU ecosystem.
[00:55] So, you think about stock market today, right?
[00:57] When you try to buy and sell stocks, you can call brokers.
[01:01] Obviously, you can call a few banks to say, "Hey, I want to buy and sell X amount of Tesla or actually SpaceX, right?"
[01:08] And you can do that or you can transact through your app at NYSE or NASDAQ.
[01:14] Doesn't mean you get the best price at NYSE or NASDAQ just because it's more efficient.
[01:19] And you know the liquidity is going to be there.
[01:21] You know the trade will be settled.
[01:23] It'll be cleared.
[01:24] You can get what you're looking for at the prices you're looking for in a very efficient manner, right?
[01:29] For capital markets and resource allocation, it's about efficiency.
[01:34] So, in computer world, obviously now we have a huge proliferation, which is great.
[01:40] We have over 300 new cloud private provider globally.
[01:43] We have a lot of token factories.
[01:45] We have a lot of API routers, which is all good for end customers.
[01:50] But at the same time, introduce a very interesting variable is how do we make transaction efficient, right?
[01:58] Do you want to call 300 sales people?
[02:00] I mean, it's great to make friends, right?
[02:02] You want to talk to them regular basis.
[02:04] They know a lot of things.
[02:05] But same time, when you need compute in the next 48 hours on reserved contracts, is easiest way is go to exchange and then go to the verified computer resources.
[02:16] Just like go to Nasdaq and I see to buy stocks.
[02:19] Right?
[02:20] So, this one of the things computing does extremely well is we have this neutral platform.
[02:25] And by design, we do not own GPUs.
[02:29] Once we start owning GPUs, that would translate to just like Nasdaq owning stocks, translate to I have a conflict of interest with you and then with the providers.
[02:38] I could cherry-pick clients, right?
[02:41] I could hoard relationships.
[02:43] Because Compute Change does not own anything by design, we work really well with all our providers globally.
[02:51] Um and one very important thing is we treat every single transaction individually, meaning we want to find you as an customer the most suited computer resources.
[03:05] And we focus on reserved compute forward contracts as well as token packages.
[03:10] So, give you some um high-level.
[03:16] So, today we have over 100 verified providers um and over 80,000 GPU in supply.
[03:25] We cover all regions, obviously ex-China.
[03:28] [gasps]
[03:29] Um our commitment is 24-hour turnaround.
[03:33] The difference between us and other so-called brokers is we do verify GPUs.
[03:39] If it's existing reserved capacity, meaning things are actually in a functional stage, we'll verify the UUIDs, the inventories, the performances.
[03:49] We run Nvidia DGX, Inference X, MLPerf, make sure you get resources you promised to get.
[03:56] Um so we're in the middle middle sort of trust the platform.
[04:00] So if it's forward capacity, I'd say you need GPUs starting Jan 1st, 2027.
[04:06] We actually look through the purchase orders, all the permits they get, make sure, you know, we help you do the due diligence level.
[04:14] And on the flip side, right?
[04:15] If you're a new cloud, you can send this this is a zillion clients looking for compute.
[04:20] How do you know who has the ability to pay at a scheduled rate?
[04:25] That's where Compute Exchange comes in.
[04:26] We do KYB KYC of the clients.
[04:29] Obviously, you want to make sure your clients is the clients you think they they are.
[04:33] So we step in and do that kind of verification just like any exchange would do.
[04:38] And everything we do in the forward markets is actually a precursor to physically delivered futures.
[04:45] Also give you some context why I am what I do.
[04:47] Sorry, I probably shouldn't say that.
[04:49] I start my career as energy option market maker um at DRW Trading, Chicago.
[04:54] So I did that for 4 5 years.
[04:56] So as option energy market maker, everything's very intuitive.
[05:00] You have WTI, you have Brent, right?
[05:02] WTI is a physically delivered commodity contracts.
[05:06] Brent is cash settled.
[05:08] They also have very distinct use cases.
[05:11] You're looking for hedging in the financial sense, right?
[05:15] The your price volatility, your cost volatility, you're going to Brent.
[05:19] You're looking physically procure resources, use WTI.
[05:23] So we're creating that the WTI at Compute Exchange.
[05:30] So the ways to buy compute through um Compute Exchange right now, you can get reserved contracts, you can get forward contracts on both GPU and token.
[05:39] Let me explain.
[05:40] The forward contracts on GPU is very reasonable, right?
[05:43] You want something starting in the future, nothing available right now.
[05:46] So that's something we're doing on regular basis.
[05:49] Token forward I think can be a newer-ish concept.
[05:52] Basically, you can think about as a dedicated deployment, on-premise deployment, right?
[05:58] Similarly, you reserve a bare metal GPU clusters.
[06:02] You reserve a cluster, but you have dedicated token deployment on top of that.
[06:07] So, you get guaranteed SLAs for that particular models.
[06:11] You get quantization, precision, everything's fixed for you for a period of time.
[06:17] So, we try to solve the on-demand token API volatility pricing problem here.
[06:24] The third is we have a huge market, open hardware market.
[06:28] It's really interesting.
[06:29] The secondary server transactions started last year when people just pinging me say, "Hey Carmen, I have like 20,000 H100 sitting at data center.
[06:40] Can you take away from me?"
[06:41] So, I was like, I don't know.
[06:43] I can't store it in my home, obviously.
[06:45] So, we started having a marketplace for secondary server sales.
[06:49] This has been a very vibrant market for the past I would say 12 months.
[06:54] We cover again all servers, even B200 you see secondary transaction is happening.
[07:00] Sometimes trading above MSRP, right?
[07:03] Can make sense think about like COVID times when secondary cars trading above, right?
[07:08] It's a brand new car price.
[07:11] So, we covered the three major layers of compute.
[07:17] So, reserve GPU, obviously, I think the previous speaker covered extremely well with the terms.
[07:23] The minimum we do is one node starting at one month, and then we can go all the way to three, four, five years, right?
[07:29] It's all depends on the availability, the providers, but you get a chance to state your preferences.
[07:36] We don't do on-demand.
[07:40] And another thing really works really well for config changes we do provide a flexibility.
[07:46] Let's say you have over committed it in terms of quantity or duration, you can sell it back to config change, right?
[07:52] So, that's something we committed to do.
[07:54] So, you don't have to worry too much if you over bought.
[08:00] All right, segment life So, I will actually do a quick demo later.
[08:04] Um so, we have a lot listings, marketplace listings you can browse and chat with the providers and figure out the best silver mechanisms.
[08:12] You can also RFQ if you don't see anything you like you liked.
[08:15] We actually seen over a billion RFQ'd every month.
[08:21] Sounds big, but we all know it's full computer it's not that huge, right?
[08:25] But still you see the the the the the activities, which a lot is we see a big range of RFQ requests from one node all the way to 20,000
[08:36] nodes.
[08:37] So, our clients are you know, all different shapes and forms, different use cases, which is great.
[08:45] Yeah, so you set timing submit and we route it.
[08:47] So, in the behind the scene, we have inventory tracking.
[08:51] So, with over 100 providers globally, we know who has one roughly one right before capacity.
[08:57] And then I say you you submit a request in that hey, I need 30 300 GPUs B200 US East starting Jan 1st.
[09:05] We say, you know, we have three providers we think they have those.
[09:09] We'll route them to the three.
[09:10] They will respond to you through the platforms and they'll give you um a rubrics back.
[09:15] Pricing, specs, SLAs, we benchmark them if it's existing servers and then it's your choice, right?
[09:22] Price is not only indicator.
[09:24] We believe give you as much transparency as we can and then you will make a decision.
[09:32] So, everything is in one place.
[09:33] Again, the rubrics come back.
[09:34] You will see everything and we introduce you to each other.
[09:37] We never hold relationships, right?
[09:39] We'll make sure you know who you're dealing with.
[09:42] Um so, buy tomorrow's computer today's price.
[09:45] So, it's a physically procured platform.
[09:49] This is the used refurbished uh GPUs.
[09:53] So, again, we have a extremely um exciting uh ecosystem participants in this uh in this uh secondary GPU cells.
[10:02] We cover from 490s to B1200, AMD, everything.
[10:10] Inventory is life and we do connect with you through all different channels like email, chat, however you want to do this.
[10:18] Um we track end to end.
[10:20] We actually have scoring for everybody.
[10:23] We score you, our customers.
[10:25] We score our providers.
[10:27] You can't see it quite yet, but you know, it'll be really fun.
[10:30] Um we're going to have end of year celebration for our providers.
[10:33] We can We're going to give all the best provider in APAC, best provider in Europe, best provider in US, most responsible providers.
[10:42] So, we're going to have a lot of a provider awards coming up.
[10:44] Stay tuned.
[10:48] Um verify network.
[10:49] So, everything's uh you know, verified, which is you know, negotiate state exchange.
[10:54] So, we help in terms of be the middleman.
[10:57] Sometimes things are not very clear.
[10:59] We try to make sure people understand what's the best practices we have seen in the ecosystem.
[11:07] I'm going to quickly show you guys the platform.
[11:11] I think that's more fun.
[11:19] Uh here.
[11:22] So, this is the platform.
[11:24] You can see a bunch of available inventories.
[11:26] By the way, anyone can come sign up.
[11:28] It's completely free.
[11:28] There's no downside using us.
[11:30] You don't have to buy, you can browse.
[11:33] You can request for quote if you, you know, interested something we don't have.
[11:37] You can start with your vendor type.
[11:39] Again, we agnostic of GPUs.
[11:42] You can select your model, you can select your deployment type.
[11:45] You can select number of GPUs.
[11:48] You can select your compute interconnect preferences, CPU platforms, storages, everything you need.
[11:55] And then some optional items with system software stacks, regions.
[11:59] Obviously, we encourage you to select multiple regions if you open to that.
[12:02] You can add head nodes, tell us your use cases, anything you want, and then submit RFQ.
[12:09] So, this has been extremely robust.
[12:11] And then we, you know, we obviously we want to make sure um if you have any feedback, let us know.
[12:18] So, I'm going to pause on Compute Change side.
[12:20] Um I think I have some time left, which I'd love to show you some of numbers we've seen in terms of term structure.
[12:29] Also, context.
[12:31] I'm CEO for Compute Change.
[12:32] I'm also CEO for another company called Silicon Data.
[12:35] So, I'm a founder CEO for Silicon Data, which is the Bloomberg for compute.
[12:40] So, we provide indices for CME to settle their futures on, which will be happening in a few weeks.
[12:47] We I'm very excited about this.
[12:49] So, the one that's going to be start trading at CME will be the H100 near cloud on demand rate as well as V200 um prices.
[13:00] What I want to show you is implied forward curve.
[13:03] So, as I said, Compute Change we do a lot of forward capacities.
[13:07] So, essentially a lot of longer term contracts is going to be very critical using about even in your cloud or your your consumers.
[13:16] Let me just make it larger.
[13:18] So, the yellow line was last year, right?
[13:21] The the X horizontal line is the duration, and the dollar amount is the Y axis.
[13:28] So, last year was complete downward sloping curve for the yellow line, right?
[13:32] Meaning the longer term contract you signed, the cheaper you know, was per shift per hour.
[13:38] And this year, starting early this year, the whole curve shift up and became contango.
[13:43] If you see the the red line, right?
[13:45] So, translate to people not getting a discount enough discount for longer term contract.
[13:52] Implies that people assume the price going to rise down the line.
[13:56] And now you see the price the whole curve keeps shifting up, got flattened, right?
[14:01] So, meaning people are now either get penalized or get, you know, awarded by signing long term contract.
[14:08] This is reserved contract we observe.
[14:10] And if you look at implied forward rates, this is not this is the implied forward rates, right?
[14:15] Because we don't have a future trading.
[14:17] So, this is calculated based on our term structure.
[14:21] And we use similar methodology as Treasury futures curve.
[14:24] So, you can see a market consensus view is the price could go up next in the the future next 36 months.
[14:32] Again, market consensus doesn't mean it's right.
[14:36] If you look at B200, again, last year was contango and this year the whole curve shift up.
[14:42] Every single term is more expensive and it's got flattened, right?
[14:47] Implied forward rates, similarly, people think the price going to go up.
[14:51] I want to point this out the term structure how is this shows you that GPU is behaves just like any commodity, have a very clear term structure risks, which data futures are excellent at hedging.
[15:05] So, we have seen a lot block trading happening around the term structure.
[15:10] So, you think about this, right?
[15:11] If you have servers on your balance sheet, you have financing obligation for next 5-6 years, and you have reserved contracts for first three, then you're exposed essentially year four, five, or six, right?
[15:24] So, you want to hedge that risk out by entering, you know, a swaps or block trading.
[15:30] Um so, those are things Oh, or there there are trades happening even for the first three years, which you have resale contracts on.
[15:38] What if the resale contract defaulted, right?
[15:41] What's your um guarantee from some of the players?
[15:45] So, we saw block trading happening around the indices.
[15:48] Trade as a floating rate.
[15:54] This chart um actually is being uh used differently by the NVIDIA team.
[16:02] Um in NVIDIA's latest uh um September investor day deck, they said us with only data provider that actually cited in our deck.
[16:12] Um this is very interesting.
[16:14] Silicon data becomes the appraisal for a few banks when they have GPU servers on their balance sheet.
[16:20] Um this everything you see here uh forward-looking view, right?
[16:24] So, it's discounted cash flow.
[16:27] So, as I said, we have a time structure.
[16:29] So, assume A100s, right?
[16:31] A100s have been around for six years.
[16:33] If today you sign a three-year long lease on A100s, and you have a rate, and we discount the the cash back to time zero, you have how much the asset's worth,
[16:44] right?
[16:45] Obviously, there's a lot of things baked in, utilization rate, what is your cost.
[16:50] There's a bunch of assumptions which we're pretty transparent about.
[16:54] Um we actually calculate the residual value for normalized value, obviously, across all A100, H100, B200 servers globally.
[17:03] And then we'll compare that with the straight-line depreciation, which is everybody using the five-year depreciation.
[17:09] What's interesting is the A100, right?
[17:11] Even though it is downward sloping, you know, think means things are a but it's worth the accounting depreciation curve, straight line.
[17:20] And the H100 obviously, you know, it's going going bananas now and the price actually went up.
[17:26] My My analogy would be just like secondary cars during COVID time.
[17:31] The price went up, right?
[17:33] And B200 obviously went above its MSRP by this point.
[17:38] So, this is give some give you some views about what's going on in the secondary market side.
[17:42] I think I'm out of time, but you know, feel free to follow me at Carmen CLI at computerhistory.com, CLI@computerhistory.com.
[17:51] Feel free to get me on Twitter.
[17:52] Carmen Li is my Twitter handle.
[17:54] Just Carmen Li.
[17:56] That's it.
[17:57] Thank you.