WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=5dCAmSDOAjI NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/5dCAmSDOAjI.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:01.309 --> 00:00:03.309 [music] s2 00:00:12.480 --> 00:00:13.280 Hi everyone. s3 00:00:13.280 --> 00:00:14.480 Uh welcome to the talk. s4 00:00:14.480 --> 00:00:16.560 First of all, thank you all for making the time. s5 00:00:16.560 --> 00:00:22.400 I know it's the last talk of the day probably or I think the last one is at uh 3:45, but uh yeah, thank you all for your time. s6 00:00:22.400 --> 00:00:24.235 I I know you're all probably very busy. s7 00:00:24.235 --> 00:00:24.560 [snorts] s8 00:00:24.560 --> 00:00:31.840 Um, today I'm going to be talking about something that is a little bit slightly futuristic, though not for San Francisco, and that's world models. s9 00:00:31.840 --> 00:00:38.239 Um, I know here it's written real-time interactive video, but the way we think about world models is really in real time interactive video. s10 00:00:38.239 --> 00:00:39.360 And I'll explain why. s11 00:00:39.360 --> 00:00:47.600 Um, and in today's world, I think world models is a little bit of a marketing term that people think about it from a gosh and splatting standpoint, others from video. s12 00:00:47.600 --> 00:00:51.920 Uh, but the way we define world models is really real- time interactive video. s13 00:00:51.920 --> 00:00:59.760 And I have strong evidence or like beliefs that this will actually change everything in how we produce and consume content. s14 00:00:59.760 --> 00:01:01.600 Oops. s15 00:01:01.600 --> 00:01:03.120 What happened? s16 00:01:03.120 --> 00:01:05.120 Sorry. s17 00:01:06.320 --> 00:01:08.799 Sorry about that. s18 00:01:09.760 --> 00:01:10.740 Cool. s19 00:01:10.740 --> 00:01:14.720 [clears throat] So if you think about video, video has always been something passive. s20 00:01:14.720 --> 00:01:19.360 Before people used to produce videos and movies and then we would watch it. s21 00:01:19.360 --> 00:01:21.360 And in today's world we have these models. s22 00:01:21.360 --> 00:01:28.000 the V3, the C dense 2 and what they do is you prompt them, you get back a file, you watch and good luck. s23 00:01:28.000 --> 00:01:28.960 It's a slot machine. s24 00:01:28.960 --> 00:01:31.920 You cannot change it, you cannot do anything about it. s25 00:01:31.920 --> 00:01:43.600 So there's a question that um we like to think about in our company is what happens when video becomes programmable like software and what happens when pixels can be generated in real time. s26 00:01:43.600 --> 00:01:49.119 Once this happens, it actually changes completely how we think about consuming content and even producing content. s27 00:01:49.119 --> 00:01:58.960 and I'll be talking about how in history we've seen that real time has always been the future and the kind of applications that it unfolded and how you can get started today. s28 00:02:00.719 --> 00:02:02.240 Quick quick background here. s29 00:02:02.240 --> 00:02:03.360 Um so my name is Ahmed. s30 00:02:03.360 --> 00:02:05.200 I'm the head of go to market at reactor. s31 00:02:05.200 --> 00:02:07.600 My background is in computer vision and machine learning. s32 00:02:07.600 --> 00:02:12.080 Um and I state machine learning because this is the time where we used to actually train our models. s33 00:02:12.080 --> 00:02:16.720 Um I built and shipped games on iOS uh on iOS and Android for fun. s34 00:02:16.720 --> 00:02:18.000 Um I was a founder. s35 00:02:18.000 --> 00:02:28.000 Um and today I'm the head of go to market at reactor and who we are quickly is we're a series A company building the platform for these real-time world models. s36 00:02:28.000 --> 00:02:42.560 So so far the main most of them have been still models but we are building the infrastructure and the developer platform to make them usable so that anyone can integrate these real-time interactive video and we can democratize access to this technology. s37 00:02:43.440 --> 00:02:44.879 I'll start with a problem. s38 00:02:44.879 --> 00:02:48.720 We can today generate pretty much anything, but we just can't change it, right? s39 00:02:48.720 --> 00:02:52.000 A generator video, as I mentioned earlier, is still a recording. s40 00:02:52.000 --> 00:02:55.120 You get it back, you can't do anything about it. s41 00:02:55.120 --> 00:02:57.120 And real time changes what the medium is. s42 00:02:57.120 --> 00:02:59.360 It doesn't just make it faster. s43 00:02:59.360 --> 00:03:04.720 And I'm going to talk about two examples that actually show us what real time has unlocked in the past. s44 00:03:04.720 --> 00:03:10.319 Before, in the 1950s and even before, we used to look at a map to know where we are. s45 00:03:10.319 --> 00:03:12.720 Someone produces a map, you look at where you are, and that's it. s46 00:03:12.720 --> 00:03:14.400 You cannot do anything about it. s47 00:03:14.400 --> 00:03:15.599 Then GPS came. s48 00:03:15.599 --> 00:03:17.120 GPS made it real time. s49 00:03:17.120 --> 00:03:21.360 Suddenly I can know where where I am at the instant and I can you know track it. s50 00:03:21.360 --> 00:03:24.400 Now you think GPS has just made it a bit faster to know where I am. s51 00:03:24.400 --> 00:03:30.000 But actually Uber would not exist if we did not have the GPS. s52 00:03:30.000 --> 00:03:32.400 Another example which is even a bit more powerful. s53 00:03:32.400 --> 00:03:37.440 I'd say before we used to to use the film to produce content, right? s54 00:03:37.440 --> 00:03:38.400 We had a film. s55 00:03:38.400 --> 00:03:41.519 Someone shoots something they can't see what they're shooting. s56 00:03:41.519 --> 00:03:45.040 They go somewhere, they produce that film, and then you can see it. s57 00:03:45.040 --> 00:03:47.040 And then it became digital. s58 00:03:47.040 --> 00:03:48.799 You can start seeing what you're shooting. s59 00:03:48.799 --> 00:03:54.480 If today you pick up your iPhone and you start recording a video, then you can see what's going on. s60 00:03:54.480 --> 00:03:56.879 And that's why we can produce high quality content. s61 00:03:56.879 --> 00:04:01.280 It's because you are able to see what's going on in the screen and adapt accordingly. s62 00:04:01.280 --> 00:04:04.799 That gave rise to in Instagram and Tik Tok. s63 00:04:04.799 --> 00:04:08.480 Instagram and Tik Tok would not exist if we could not produce high quality content. s64 00:04:08.480 --> 00:04:15.200 And the only reason why we're able to produce high quality content among many reasons is because we can see in real time what's happening. s65 00:04:15.200 --> 00:04:16.560 It's not a slot machine. s66 00:04:16.560 --> 00:04:19.440 We can actually just edit and see. s67 00:04:19.440 --> 00:04:23.280 And that's what unlocks all of these new use cases. s68 00:04:24.240 --> 00:04:27.040 So when video becomes programmable, you can address it. s69 00:04:27.040 --> 00:04:28.080 You can condition it. s70 00:04:28.080 --> 00:04:29.840 You can change it. s71 00:04:29.840 --> 00:04:32.800 I can show you on the screen whatever I want to show you on the screen. s72 00:04:32.800 --> 00:04:39.360 And it becomes programmable a little bit like software or like anything that is programmable in the world. s73 00:04:39.600 --> 00:04:44.000 And in in the market today we are seeing three kinds of models that do this. s74 00:04:44.000 --> 00:04:48.080 Some of them you will be familiar with others you will not be familiar with. s75 00:04:48.080 --> 00:04:56.479 The first one is these are VO think about VO or Sora but real time and interactive meaning that first of all they're infinite. s76 00:04:56.479 --> 00:05:00.080 So they don't stop after 5 10 or 30 seconds. s77 00:05:00.080 --> 00:05:02.720 They actually continue forever. s78 00:05:02.720 --> 00:05:09.680 They're interactive, meaning you can change what's happening on the screen, and they're in real time, so you don't need to wait to see what's going on. s79 00:05:09.680 --> 00:05:17.199 And assuming this works, so this is an example of a of a video that I passed an image with a dog, and this was all generated in real time. s80 00:05:17.199 --> 00:05:22.720 And at some point, I'm going to prompt a cat shows up and you will see that a cat showed up in the in the video. s81 00:05:22.720 --> 00:05:30.000 This would not be possible in the existing batch regular video generation models because you would get back the video and you cannot do anything about it. s82 00:05:30.000 --> 00:05:31.280 I could have added anything. s83 00:05:31.280 --> 00:05:33.600 I could have gone on to create an entire story with it. s84 00:05:33.600 --> 00:05:40.320 I could have said the cat the dog starts running, starts jumping, a dragon shows up, it goes to I don't know to the world cup. s85 00:05:40.320 --> 00:05:43.600 All of this would have happened in front of you. s86 00:05:43.840 --> 00:05:47.919 These type these types of models pro like unlock a few things. s87 00:05:47.919 --> 00:05:49.360 First of all, control. s88 00:05:49.360 --> 00:05:55.759 So if you think about generative media today, the big problem that content creators all have is I don't have the control I need. s89 00:05:55.759 --> 00:06:01.440 Like yes, it's great to use Cense 2 or VO3 to generate videos, but I just don't have the control. s90 00:06:01.440 --> 00:06:07.680 And this is always the thing that any filmmaker or movie producer or any content creator will tell you. s91 00:06:07.680 --> 00:06:14.319 And so real time actually ends the slot machine type of mentality and actually gives you the the control that you need. s92 00:06:14.319 --> 00:06:19.520 And a big saying I like to say is instant feedback is the ultimate level of control. s93 00:06:19.520 --> 00:06:23.919 And you we will never be able to have this level of control if we don't have real time. s94 00:06:23.919 --> 00:06:33.039 The second thing it unlocks, which you know, among other things, which is a field I'm not particularly fond of, but I think is going to be big, is advertising. s95 00:06:33.039 --> 00:06:39.120 If I can know what you looked for a minute ago, why can't I insert the logo of whatever you've been looking for? s96 00:06:39.120 --> 00:06:41.840 Why can't I produce an ad in real time in front of you? s97 00:06:41.840 --> 00:06:44.000 We don't need to produce pre-produce anything. s98 00:06:44.000 --> 00:06:55.600 Now granted, this is going to take some time because you know brands afraid of AI, afraid of uh you know if their logo has one pixel that is white instead of dark s99 00:06:55.600 --> 00:07:02.080 but it will happen eventually and so and I think at the moment that happens we will not be to need to produce any ads anymore. s100 00:07:02.080 --> 00:07:06.000 Everything will be happening in front of you in real time. s101 00:07:06.000 --> 00:07:09.440 The second type of model is the one that probably you're most familiar with. s102 00:07:09.440 --> 00:07:12.000 It's the Genie3 like from Google. s103 00:07:12.000 --> 00:07:18.800 These are models where you can pass an image and a text typically and you can control a character. s104 00:07:18.800 --> 00:07:19.680 They're fun. s105 00:07:19.680 --> 00:07:22.960 The first thing that you think about when you think about this is games, right? s106 00:07:22.960 --> 00:07:23.759 It's a character. s107 00:07:23.759 --> 00:07:24.400 It's a world. s108 00:07:24.400 --> 00:07:26.319 You can generate anything. s109 00:07:26.319 --> 00:07:32.400 But actually, it actually goes way far way beyond games. s110 00:07:32.400 --> 00:07:42.800 It creates entire new interactive experiences where combined with the first types of models that I talked about, we've already been seeing people in our community building a mix of games and and movies. s111 00:07:42.800 --> 00:07:57.840 If you've ever watched Benders Nash from Netflix, which is the game, the movie that you can pick your next scene, this is one of those things that gets uh that that becomes possible that you can control a character, you can control what's happening and create entire new types of interactive experiences s112 00:07:57.840 --> 00:08:00.479 that were not possible. s113 00:08:00.639 --> 00:08:03.120 The second thing is robotics. s114 00:08:03.120 --> 00:08:07.840 So because you can simulate and you can control, you can actually create as much training data as you want. s115 00:08:07.840 --> 00:08:12.000 And robotics, world models in robotics is actually a ginormous market today. s116 00:08:12.000 --> 00:08:17.440 Um I cannot tell you the number of robotics labs that um are training and building these models. s117 00:08:17.440 --> 00:08:26.560 But because you can control whatever you want to control in any environment, this creates a new opportunity to generate infinite amount of data for robotics. s118 00:08:26.960 --> 00:08:32.080 And finally, something I like to think about, this is more maybe a passionate thing that I have is education. s119 00:08:32.080 --> 00:08:42.959 Um, because you can step into anything who with today's world in AI, I don't actually believe that educ in the future of education is LLM based or textbook based. s120 00:08:42.959 --> 00:08:55.360 If you can put any any kid in the situation, for example, in a history lesson, that enables entire new types of educ experiences that uh can be educational. s121 00:08:56.240 --> 00:09:06.480 The third type of model is probably the type of model that you know is more, let's say, um, something that we've been seeing before, which is avatars, but live and interactive. s122 00:09:06.480 --> 00:09:10.240 The thing with avatars, though, is it hasn't actually been cracked. s123 00:09:10.240 --> 00:09:11.680 They're still all kind of weird. s124 00:09:11.680 --> 00:09:16.320 If you speak to an avatar in any customer support or anything, it's still kind of off, right? s125 00:09:16.320 --> 00:09:26.000 Um, and these types of models, and we're seeing a rise of these live and interactive avatar models in research preview, that combined with model one and model two is actually going to be, s126 00:09:26.000 --> 00:09:30.240 I believe, a a big change in what we've been seeing so far. s127 00:09:30.240 --> 00:09:44.560 And this will be applied to things like customer support, training, sales, gaming, streaming services, etc. And so just just to give you a glimpse of what our users are building today at Reactor with these types of models, s128 00:09:44.560 --> 00:09:47.200 some of them are building interactive live stream, right? s129 00:09:47.200 --> 00:09:52.560 A live stream where people are watching and then the users can type what happens next and then they vote. s130 00:09:52.560 --> 00:09:53.440 Why? s131 00:09:53.440 --> 00:09:55.279 Because pixels can be generated in real time. s132 00:09:55.279 --> 00:10:02.240 So there's no reason why I cannot put a live stream on X, YouTube or Twitch and enable users to pick what happens next. s133 00:10:02.720 --> 00:10:06.000 Something that was surprising to me is a little bit on on the medical simulation. s134 00:10:06.000 --> 00:10:12.080 So we've seen users create applications where you generate a world and then you simulate what happens next. s135 00:10:12.080 --> 00:10:13.760 What if I put this medicine? s136 00:10:13.760 --> 00:10:15.279 What if I remove this medicine? s137 00:10:15.279 --> 00:10:15.600 Right? s138 00:10:15.600 --> 00:10:21.839 And this creates can be a training playground for people wanting to become doctors. s139 00:10:21.839 --> 00:10:25.279 The third one which is also kind of surprising is cooking simulation. s140 00:10:25.279 --> 00:10:32.640 So people are building applications where you can simulate cooking and what happens if you put this ingredient. s141 00:10:32.640 --> 00:10:34.720 And finally, video editing. s142 00:10:34.720 --> 00:10:42.640 With videotovideo models, video editing becomes very interesting because I'm able to just add visual effects in real time. s143 00:10:42.640 --> 00:10:46.160 And we've seen people build entire video edit editing platforms. s144 00:10:46.160 --> 00:10:57.200 Now, granted, they're not very good yet just because of the quality of the models, but it's a new paradigm when you're able to edit videos just via prompting or via talking to it or via clicking. s145 00:10:58.959 --> 00:11:02.560 And now for the final part, how do you actually do all of this? s146 00:11:02.560 --> 00:11:06.560 And this is why I like to save the world behind an API. s147 00:11:07.040 --> 00:11:09.920 At at Reactor, we have four types of models today. s148 00:11:09.920 --> 00:11:13.440 First one is called Helios, which is the interactive video model that I talked about. s149 00:11:13.440 --> 00:11:15.600 This one is from Bite Dance. s150 00:11:15.600 --> 00:11:20.640 Lingbot, which is a world model like Genie 3 trained by Alibaba. s151 00:11:20.640 --> 00:11:24.560 Long live 2 from Nvidia, which is multi-shot film. s152 00:11:24.560 --> 00:11:34.560 You can prompt things in advance and create a consistent story over time and sound streaming also from Nvidia which is a model that does video to video editing. s153 00:11:34.560 --> 00:11:47.040 So people are already using this for example by shooting something or creating something on CDE 2 uploading it and then adding visual effects removing people adding background and this be gets very interesting in previsualization s154 00:11:47.040 --> 00:12:06.320 for example for Hollywood movies and under the hood when we talk about infrastructure the thing that I think I like to drive home is building infrastructure for regular video generation models is very different from real time because in regular video generation models, you're talking about requests. s155 00:12:06.320 --> 00:12:14.639 You just send a request, a job gets run in the cloud, and I'm oversimplifying here, but you know, a job runs in the cloud and it gives you back a file. s156 00:12:14.639 --> 00:12:17.120 With real time, it's a different ballgame. s157 00:12:17.120 --> 00:12:22.880 Uh you cannot just take what works for batch inference and apply to real-time inference. s158 00:12:22.880 --> 00:12:25.760 For example, you need to think about streaming, right? s159 00:12:25.760 --> 00:12:37.360 Once you need to think about streaming um pixels uh from a server to the client um it adds entire new entire complexities that batch generation does not have to think about. s160 00:12:37.360 --> 00:12:39.839 The second one is that everything is a live session. s161 00:12:39.839 --> 00:12:43.760 So everything runs constantly and there's memory to be kept into account. s162 00:12:43.760 --> 00:12:49.200 Now granted one of the things that live vid live realtime models struggle with is memory. s163 00:12:49.200 --> 00:12:56.079 If you've seen demos from Genie, for example, we've all seen that the character can look back and then not remember what what's going on. s164 00:12:56.079 --> 00:13:05.839 So, there is a lot of work that needs to be going into maintaining that context window so that you can remember what happened if you turned your character left and right. s165 00:13:06.000 --> 00:13:08.320 And finally, global scale. s166 00:13:08.320 --> 00:13:13.760 If you if you're think about real time, it needs to be sub 100 millisecond latency anywhere you are. s167 00:13:13.760 --> 00:13:26.000 And if you're deploying applications in the world, then someone based in India or someone based in Japan should be routed to a GPU that is based in India or Japan or as close as possible to it. s168 00:13:26.000 --> 00:13:34.160 If not, if you don't have the compute worldwide, then the experiences are not real time anymore and it breaks completely the medium. s169 00:13:35.120 --> 00:13:39.839 And with Reactor, this is as easy as it gets to integrate these real-time mods. s170 00:13:39.839 --> 00:13:44.480 Okay, I kind of maybe oversimplified it a little bit here, but it really is maybe 10 lines of code. s171 00:13:44.480 --> 00:13:54.079 Um, and we have docs to do all of this, but essentially you can just load the model with an API key and just start integrating it in whatever video, image, plugin, s172 00:13:54.079 --> 00:13:56.720 anything that you're building. s173 00:13:56.959 --> 00:13:59.680 And if you want to get started, there's a QR code there. s174 00:13:59.680 --> 00:14:05.199 And I've added a promo code AIE 2026, which will give you $75 off. s175 00:14:05.199 --> 00:14:15.279 Well, $75 worth of credits, which is um significant amount of compute uh in our case because we make the models extremely cheap. s176 00:14:15.279 --> 00:14:17.839 Thank you very much. s177 00:14:18.059 --> 00:14:19.600 [applause] s178 00:14:19.600 --> 00:14:23.680 I'm happy to take questions if anybody has a question. s179 00:14:26.800 --> 00:14:28.800 Okay. s180 00:14:40.079 --> 00:14:44.399 uh well one thing that actually we do is multiGPUs. s181 00:14:44.399 --> 00:14:46.000 So we use multiple GPUs. s182 00:14:46.000 --> 00:14:51.120 Um optim optimizing the model weights applying quantization techniques. s183 00:14:51.120 --> 00:14:56.800 So there are ways it's just a matter of priorities but it is uh there are ways around it. s184 00:14:56.800 --> 00:15:02.240 Yeah good question though. s185 00:15:06.800 --> 00:15:07.279 Hey. s186 00:15:07.279 --> 00:15:09.279 Yeah. s187 00:15:15.680 --> 00:15:16.240 Sorry. s188 00:15:16.240 --> 00:15:18.560 Are you aware of what the s189 00:15:18.560 --> 00:15:20.800 uh No. s190 00:15:21.279 --> 00:15:23.279 Okay. s191 00:15:27.279 --> 00:15:29.120 I wasn't, but now I'm going to look at it. s192 00:15:29.120 --> 00:15:29.455 Yeah. s193 00:15:29.455 --> 00:15:31.455 [laughter] s194 00:15:31.680 --> 00:15:33.680 Yeah. s195 00:15:35.920 --> 00:15:38.639 How do you feel like? s196 00:15:41.839 --> 00:15:43.839 Yeah. s197 00:15:45.760 --> 00:15:46.320 Yeah. s198 00:15:46.320 --> 00:15:49.680 Do you guys experiment with that? s199 00:15:49.680 --> 00:15:53.600 When you say deterministic engines, what do you mean exactly? s200 00:15:53.600 --> 00:16:00.560 any kind of like deterministic rule set that you check against maybe so that the simulation stays s201 00:16:03.920 --> 00:16:12.800 so we don't do any of that today um the reason why also we build a developer platform and but we've seen people build that on top of us um s202 00:16:12.800 --> 00:16:21.120 building the infra for this is already a lot of work um and I think we we were seeing already developers building and then open sourcing and then we reuse it s203 00:16:21.120 --> 00:16:24.320 um but so the community is doing it for which is even better. s204 00:16:24.320 --> 00:16:26.320 Yeah. s205 00:16:26.320 --> 00:16:26.720 Question. s206 00:16:26.720 --> 00:16:28.720 Yeah. s207 00:16:33.519 --> 00:16:38.560 Uh you're asking a question that the entire research community in world models has not answered. s208 00:16:38.560 --> 00:16:42.800 You're evolves for real time and consistency is uh and fidelity. s209 00:16:42.800 --> 00:16:43.839 Well, fidelity is easy. s210 00:16:43.839 --> 00:16:45.279 It's just like pixels, right? s211 00:16:45.279 --> 00:16:50.480 But um evaluation for these real-time models is an unsolved problem. s212 00:16:50.480 --> 00:16:55.680 So today it's literally just look at it and human judgment. s213 00:16:55.680 --> 00:16:57.199 That's what it is today. s214 00:16:57.199 --> 00:17:00.320 And this is including by the way J deep mind and everything. s215 00:17:00.320 --> 00:17:03.440 Nobody has solved this problem yet. s216 00:17:04.240 --> 00:17:05.039 We're working on it. s217 00:17:05.039 --> 00:17:06.079 We have a research team. s218 00:17:06.079 --> 00:17:06.396 Yes. s219 00:17:06.396 --> 00:17:08.396 [laughter] s220 00:17:11.039 --> 00:17:11.520 Awesome. s221 00:17:11.520 --> 00:17:11.760 Cool. s222 00:17:11.760 --> 00:17:12.559 Well, thank you everyone. s223 00:17:12.559 --> 00:17:15.039 Thanks for your time.