WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=apyrzaWj0Z4 NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/apyrzaWj0Z4.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:12.559 --> 00:00:15.599 Hi everyone, thank you for joining me today. s2 00:00:15.599 --> 00:00:18.880 I hope everyone is having a great conference so far. s3 00:00:18.880 --> 00:00:20.800 Um, so my name is Dylan Kuzan. s4 00:00:20.800 --> 00:00:24.320 I am a developer relations engineer at Quadrant. s5 00:00:24.320 --> 00:00:29.119 And the title of my talk today is the frontier is coming home. s6 00:00:29.119 --> 00:00:30.960 So what do I mean by that? s7 00:00:30.960 --> 00:00:36.640 And somewhere in the next few years, something smarter than all of us is going to exist. s8 00:00:36.640 --> 00:00:41.520 We've spent years asking when, but I think when is kind of the boring question. s9 00:00:41.520 --> 00:00:47.840 I think the real one is who it belongs to because there are two versions of this. s10 00:00:47.840 --> 00:00:54.559 In one, super intelligence lives in someone else's data center and you pay by the token. s11 00:00:54.559 --> 00:00:56.160 In the other, it's yours. s12 00:00:56.160 --> 00:00:57.760 It learns from your life. s13 00:00:57.760 --> 00:01:04.320 your context stays private and nobody can switch it off, throttle it or take it away from you. s14 00:01:04.320 --> 00:01:07.360 That second version is what I want to talk about today. s15 00:01:07.360 --> 00:01:10.479 That's the frontier coming home. s16 00:01:10.960 --> 00:01:12.799 Quick thought experiments. s17 00:01:12.799 --> 00:01:18.080 What would you build if your assistants actually remembered everything you ever taught it? s18 00:01:18.080 --> 00:01:22.960 Every preference, every correction, every dead end for years. s19 00:01:22.960 --> 00:01:24.880 That's not a chatbot any anymore. s20 00:01:24.880 --> 00:01:26.720 That's a second mind. s21 00:01:26.720 --> 00:01:34.000 But the the assistant you're using right now forgets almost all of it the second the conversation ends. s22 00:01:34.000 --> 00:01:36.159 Every session starts from zero. s23 00:01:36.159 --> 00:01:38.720 We call these things intelligence. s24 00:01:38.720 --> 00:01:44.640 And they are they're just geniuses with no long-term memory. s25 00:01:44.640 --> 00:01:46.640 This talk is about fixing that. s26 00:01:46.640 --> 00:01:50.720 Not with a bigger prompt with actual memory. s27 00:01:51.040 --> 00:01:56.560 That genius used to only exist behind someone's API in a data center you never see. s28 00:01:56.560 --> 00:02:00.399 Now it runs on a on hardware that most of us own. s29 00:02:00.399 --> 00:02:04.799 Not the absolute frontier, I'm going to be honest, but shockingly close. s30 00:02:04.799 --> 00:02:12.400 A machine under $2,500 today can run what was basically last year's frontier. s31 00:02:12.400 --> 00:02:15.680 And open weights models are closing on fast. s32 00:02:15.680 --> 00:02:19.280 The gap is shrinking every quarter. s33 00:02:19.520 --> 00:02:32.640 You'd expect this to feel like a landmark uh the frontier on your desk for the price of a laptop and yet it doesn't quite land because a model on its own is just a brilliant stranger. s34 00:02:32.640 --> 00:02:34.160 It's powerful. s35 00:02:34.160 --> 00:02:36.640 It's but it it isn't yours. s36 00:02:36.640 --> 00:02:44.319 What make what makes it yours is everything it knows about you and that part hasn't come home yet. s37 00:02:44.560 --> 00:02:52.879 So you can run a frontier class model on your own hardware today and yet look at the AI stack that most of us are using today. s38 00:02:52.879 --> 00:03:04.080 The compute is rented, the models is rented, the harness is rented and the part that's supposed to become you, the thing that should compound over the years also rented. s39 00:03:04.080 --> 00:03:17.360 If we don't fix that, the most powerful technology any technology any any of us will ever touch ends up owned by whoever holds the lease, not the people it was built for. s40 00:03:17.360 --> 00:03:21.360 So let's look at what renting actually costs today. s41 00:03:21.360 --> 00:03:29.519 Starts where we starts with where the model leads because when you rent it, you don't really control it. s42 00:03:29.519 --> 00:03:32.400 This shows up in three different ways. s43 00:03:32.400 --> 00:03:40.480 Last month, a government order pulled access to two flagship models, Fable and Mythos 5 for every single customer. s44 00:03:40.480 --> 00:03:45.519 Now, compare that to weights you already that are already sitting on your desk. s45 00:03:45.519 --> 00:03:49.519 Nobody can reach into your machine and flip those off. s46 00:03:49.519 --> 00:03:53.599 Even when the model the model stays up, it doesn't stay put. s47 00:03:53.599 --> 00:03:56.959 Providers retire versions and they throttle what's left. s48 00:03:56.959 --> 00:04:04.000 When the economics get gets tight, the model you rely on can quietly get worse or vanish. s49 00:04:04.000 --> 00:04:08.239 Weights your own never change unless you change them. s50 00:04:08.239 --> 00:04:10.720 People also don't don't write prompts anymore. s51 00:04:10.720 --> 00:04:14.000 They run agents or loops for hours. s52 00:04:14.000 --> 00:04:18.720 And one task can burn thousands of times more tokens than a chat message. s53 00:04:18.720 --> 00:04:25.040 Some developers burn through over $5,000 a month of of compute on a $200 plan. s54 00:04:25.040 --> 00:04:26.400 That's a subsidy. s55 00:04:26.400 --> 00:04:30.320 And those subsidies will ultimately disappear. s56 00:04:30.880 --> 00:04:34.240 If you own the compute and the weights, the control comes back to you. s57 00:04:34.240 --> 00:04:37.759 But that's still only half the problem. s58 00:04:37.759 --> 00:04:39.840 Here's the deeper issue. s59 00:04:39.840 --> 00:04:42.240 Owning inference gives you autonomy. s60 00:04:42.240 --> 00:04:44.320 Nobody can take the model away. s61 00:04:44.320 --> 00:04:46.880 But only memory gives you continuity. s62 00:04:46.880 --> 00:04:52.960 And continuity is exactly what every major lab is trying to package up and sell back to you. s63 00:04:52.960 --> 00:04:53.919 Look at last year. s64 00:04:53.919 --> 00:05:01.680 Every Frontier Lab shipped a memory feature, not because the models got smarter, but because they don't carry anything between sessions. s65 00:05:01.680 --> 00:05:04.800 That gap is the actual product. s66 00:05:04.800 --> 00:05:14.160 A system that starts simple and keeps learning you will always feel smarter than one that starts brilliant, but forget everything about you. s67 00:05:14.160 --> 00:05:18.639 That compounding is where intelligence starts to feel personal. s68 00:05:19.280 --> 00:05:22.400 So what actually is memory? s69 00:05:22.400 --> 00:05:23.840 It's not a bigger prompt. s70 00:05:23.840 --> 00:05:26.240 It's a systems with three verbs. s71 00:05:26.240 --> 00:05:29.680 Write, retrieve, and forget. s72 00:05:29.680 --> 00:05:33.360 Same three things your brain actually does. s73 00:05:33.360 --> 00:05:35.840 And you know, there's an obvious push back. s74 00:05:35.840 --> 00:05:40.240 Why not just throw everything in a big folder of markdown files. s75 00:05:40.240 --> 00:05:46.080 And because retrieving the right memory beats dumping everything and praying the model finds it. s76 00:05:46.080 --> 00:05:49.199 Retrieval gives you something a prompt cannot. s77 00:05:49.199 --> 00:05:50.720 It is control. s78 00:05:50.720 --> 00:05:52.720 You can filter by topic. s79 00:05:52.720 --> 00:05:55.840 You can decay by recency and frequency. s80 00:05:55.840 --> 00:05:58.400 You can let relevance shift over time. s81 00:05:58.400 --> 00:06:02.160 The way human memory actually works. s82 00:06:02.160 --> 00:06:05.120 And the infrastructure has been ready for years. s83 00:06:05.120 --> 00:06:08.240 HNSW in 2016. s84 00:06:08.240 --> 00:06:16.720 Um, embeddings have been running locally since 2019, long before any any model could really use them at scale. s85 00:06:16.720 --> 00:06:20.639 The memory side of super intelligence was never the hard part. s86 00:06:20.639 --> 00:06:24.160 We were just pointing it out at ourselves. s87 00:06:25.280 --> 00:06:29.840 So today, the memory that actually knows you lands in one of two places. s88 00:06:29.840 --> 00:06:36.400 Either it's stuck in an unindexed file you can't really query or it's in someone else's cloud. s89 00:06:36.400 --> 00:06:42.560 One terms one terms of service change away from being being reshaped or moved. s90 00:06:42.560 --> 00:06:46.240 Corp Carpathy framed it in a way that really stuck with me. s91 00:06:46.240 --> 00:06:52.960 It's the model is the CPU, the memory uh the context window is the RAM and the memory is the disk. s92 00:06:52.960 --> 00:06:57.440 RAM is fast but forgets forgets the moment the session ends. s93 00:06:57.440 --> 00:07:01.199 The disk is what remembers you across every conversation. s94 00:07:01.199 --> 00:07:09.199 And right right now for almost every AI assistant on on Earth, the that disk is in someone else's data center. s95 00:07:09.199 --> 00:07:15.680 So, you know, today I'm not going to argue which memory architecture wins or is the best one. s96 00:07:15.680 --> 00:07:18.160 That's definitely a different talk. s97 00:07:18.160 --> 00:07:24.880 But the disk, the parts that holds the actual record of you should be yours fully and forever. s98 00:07:25.360 --> 00:07:30.080 So, this is the piece that I work on and it's just one way to do it. s99 00:07:30.080 --> 00:07:34.960 Um, a ve I work on a vector search engine that embeds directly into your app. s100 00:07:34.960 --> 00:07:38.560 It opens look a local store inside your process. s101 00:07:38.560 --> 00:07:40.560 You write embeddings with payloads. s102 00:07:40.560 --> 00:07:44.639 Then query offline sub millisecond time. s103 00:07:44.639 --> 00:07:48.319 Same rust score as quadrant in the cloud just running where you are. s104 00:07:48.319 --> 00:07:53.360 With quantization, a million memories can fit in less than a gigabyte. s105 00:07:53.360 --> 00:07:57.280 Small enough to live on a phone or even a Raspberry Pi. s106 00:07:57.280 --> 00:08:01.280 Close the app, come back a year later, the memory is intact. s107 00:08:01.280 --> 00:08:04.560 Every app gets its own isolated store. s108 00:08:04.560 --> 00:08:08.879 And that folder is the thing that actually follows you. s109 00:08:08.879 --> 00:08:12.479 So, I'm going to do a quick live demo here. s110 00:08:12.479 --> 00:08:20.240 Um, so right now, um, I have a video recording of a drone like going over a house. s111 00:08:20.240 --> 00:08:23.440 So, basically that drone, it's it first time starting. s112 00:08:23.440 --> 00:08:25.520 It's it's first time out in the world. s113 00:08:25.520 --> 00:08:27.680 It doesn't know anything. s114 00:08:27.680 --> 00:08:34.240 And so, this this is actually running live right now fully offline on my laptop. s115 00:08:34.240 --> 00:08:38.479 And I'm using yellow which is an object detection model. s116 00:08:38.479 --> 00:08:45.360 So it detects the object and creates a um label for every items it sees here. s117 00:08:45.360 --> 00:08:56.959 Then we take take that picture, we take that label and we turn that into embeddings to create a live realtime memory for that drone. s118 00:08:56.959 --> 00:09:02.720 And and that way we can re the drone can remember everything it has ever seen before. s119 00:09:02.720 --> 00:09:09.600 And so here you can see a 2D representation of that vector space with all the memories inside. s120 00:09:09.600 --> 00:09:16.000 And memories that are the most similar will will be the closest to each other. s121 00:09:16.000 --> 00:09:23.760 So here you can see that we've already recognized 92 different objects and we have over three 300 vectors. s122 00:09:23.760 --> 00:09:28.560 So that's 300 different representations of those 90 objects. s123 00:09:28.560 --> 00:09:37.600 And you can see that the entire memory and quadrants engine uh footprints is only 15 megabytes. s124 00:09:38.399 --> 00:09:41.600 And then what we can do uh on those memory? s125 00:09:41.600 --> 00:09:46.399 Well, we enable um semantic search over those memories. s126 00:09:46.399 --> 00:09:59.519 So, you know, if I click here on the um coffee table, you can see that in less than one millisecond, I was able to pull up every single coffee table that this drone has s127 00:09:59.519 --> 00:10:00.880 seen before. s128 00:10:00.880 --> 00:10:03.120 And so, you know, we have the image here. s129 00:10:03.120 --> 00:10:05.760 We have the first sin time stamp. s130 00:10:05.760 --> 00:10:07.839 We have the last s time stamp. s131 00:10:07.839 --> 00:10:10.399 We have a definition of that coffee table. s132 00:10:10.399 --> 00:10:16.320 and and also we can see how many times that u table has been seen. s133 00:10:16.320 --> 00:10:20.320 So this is all being run and processed locally. s134 00:10:20.320 --> 00:10:22.800 There's no network attachment at all. s135 00:10:22.800 --> 00:10:28.000 Um but we also have a capability of uh cloud sync. s136 00:10:28.000 --> 00:10:37.360 So for example, you know, if you have like a swarm of drones or robots navigating the world, they can have like a hive mind in the in the cloud where s137 00:10:37.360 --> 00:10:43.279 I you know, one system uh learns something and they can all learn about it. s138 00:10:46.160 --> 00:10:53.360 And you know, this is what this was like an example for a drone, but you can apply that to to everything. s139 00:10:53.360 --> 00:10:58.160 just a chatbot, an agent that you call to a code assistant. s140 00:10:58.160 --> 00:11:05.279 And you know, if you create those memories for just a few weeks, the stranger just disappears. s141 00:11:05.279 --> 00:11:11.760 It remembers your corrections, your preferences, the dead ends you had to only hit once. s142 00:11:11.760 --> 00:11:21.680 And a model that's merely good, but never forgets you and keeps compounding over the years starts to feel super human in practice. s143 00:11:21.680 --> 00:11:26.000 Not because the model changed, but because it never stopped learning you. s144 00:11:26.000 --> 00:11:27.440 And it's portable. s145 00:11:27.440 --> 00:11:31.519 Swap the resulting model, change the hardware, none of it matters. s146 00:11:31.519 --> 00:11:37.920 It all comes with you if you keep the same embedding model and that folder becomes permanent. s147 00:11:37.920 --> 00:11:42.959 A lifetime of context fully yours moving from device to device. s148 00:11:42.959 --> 00:11:44.959 But we can go even further than that. s149 00:11:44.959 --> 00:11:49.040 You know, this can go way past a laptop or a drone. s150 00:11:49.040 --> 00:11:51.760 points that same memory at your whole day. s151 00:11:51.760 --> 00:11:53.760 Where did I leave my badge? s152 00:11:53.760 --> 00:11:56.320 What was her daughter's name again? s153 00:11:56.320 --> 00:11:58.240 What was on that whiteboard? s154 00:11:58.240 --> 00:12:02.720 And what what was the song that was playing when we met? s155 00:12:02.720 --> 00:12:06.800 Every one of those is just a a retrieval query. s156 00:12:06.800 --> 00:12:10.959 And you know, you just watched a drone remember objects. s157 00:12:10.959 --> 00:12:15.279 No, extend that to everything you see and everything you hear. s158 00:12:15.279 --> 00:12:17.600 And you know, that's not science fiction. s159 00:12:17.600 --> 00:12:25.519 Those are the smart glasses that are already shipping today and that we have seen today in this room. s160 00:12:25.519 --> 00:12:27.360 It's not a smarter you. s161 00:12:27.360 --> 00:12:30.720 It's a you that doesn't forget. s162 00:12:30.880 --> 00:12:38.000 And right now, none of those memories share uh none of those devices share a memory that you actually own. s163 00:12:38.000 --> 00:12:44.320 They could on your own terms because Frontier models are already building an index of you today. s164 00:12:44.320 --> 00:12:45.920 They have your relationships. s165 00:12:45.920 --> 00:12:47.360 They have your habits. s166 00:12:47.360 --> 00:12:50.079 They they have your whole inner life. s167 00:12:50.079 --> 00:12:54.079 That index is common whether you opt in or not. s168 00:12:54.079 --> 00:13:00.560 The only question is whether you own it or if you're renting access to yourself. s169 00:13:01.040 --> 00:13:07.519 And one last idea because owning memory doesn't doesn't mean keeping it to yourself. s170 00:13:07.519 --> 00:13:13.600 Edge can sync parts of your store to the quad quadrant cloud out of the box when you choose to. s171 00:13:13.600 --> 00:13:18.480 Picture a family, each wearing glasses that record their own day. s172 00:13:18.480 --> 00:13:23.760 Separate memories, but they can pull into a hive mind the whole family owns. s173 00:13:23.760 --> 00:13:28.240 Mom's day, dad's day, your day, searchable together. s174 00:13:28.240 --> 00:13:29.600 Now, scale that up. s175 00:13:29.600 --> 00:13:42.399 Instead of one company's super intelligence serving billions of identical people, you get thousands of small private ones, each shaped by a life, a family, a team. s176 00:13:42.399 --> 00:13:45.360 But that same mechanism cuts both ways. s177 00:13:45.360 --> 00:13:52.880 A memory can be shared without a memory share can be shared with consent or it can be extracted without it. s178 00:13:52.880 --> 00:14:00.480 That's exactly why it has to start local and why sharing should always be optin never the default. s179 00:14:00.480 --> 00:14:06.000 You should really decide who gets your continuity and your memories. s180 00:14:06.399 --> 00:14:08.160 And this is where I'm going to leave you. s181 00:14:08.160 --> 00:14:14.160 You know, every everyone here is going to watch super intelligence show up over the next few years. s182 00:14:14.160 --> 00:14:16.720 That's that part is basically settled. s183 00:14:16.720 --> 00:14:19.360 What's not settled is who it belongs to. s184 00:14:19.360 --> 00:14:22.399 And that's the fight worth having right now. s185 00:14:22.399 --> 00:14:26.160 While the AR while the architecture is still up for grabs. s186 00:14:26.160 --> 00:14:29.920 So tonight, don't go home and just try a tool. s187 00:14:29.920 --> 00:14:32.800 Take the stranger home and give it memory. s188 00:14:32.800 --> 00:14:36.000 Then picture that memory 5 years from now. s189 00:14:36.000 --> 00:14:48.880 not a chatbot, a private mind that never forgets you, that nobody can switch off, and that knows knows you better than any system ever has because you were the only one who ever trained it. s190 00:14:48.880 --> 00:14:52.639 That's not a that's not a smaller super intelligence. s191 00:14:52.639 --> 00:14:56.160 That's the only one worth wanting. s192 00:14:56.240 --> 00:14:58.240 All right, that's all for me. s193 00:14:58.240 --> 00:15:00.800 Thank you very much. s194 00:15:14.390 --> 00:15:16.390 [music]