WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=4Gqg0HVe-AY NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/4Gqg0HVe-AY.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:01.309 --> 00:00:03.309 [music] s2 00:00:12.880 --> 00:00:13.679 Okay. s3 00:00:13.679 --> 00:00:15.065 Hi everyone. s4 00:00:15.065 --> 00:00:16.880 [applause] s5 00:00:16.880 --> 00:00:22.720 Um, and thanks a lot for making it down to the second floor and finding my session here. s6 00:00:22.720 --> 00:00:23.119 Okay. s7 00:00:23.119 --> 00:00:33.120 So I'm Chris Manning and what I want to do today is present something about Moonlakes's approach to producing a simulation infrastructure for practical physical AI. s8 00:00:33.120 --> 00:00:45.120 The overall goal here is that a north star for AI and actually cognitive science as well has always been to understand and work out how to build embodied intelligence. s9 00:00:45.120 --> 00:00:55.440 So today I'm going to tell us about some of the recent work that we've been doing at approaching a practical form of embodied artificial general intelligence. s10 00:00:55.440 --> 00:01:06.880 But you know I'm not actually going to start there because uh you know um Swick said no you shouldn't just do that. s11 00:01:06.880 --> 00:01:10.240 Um, you should have a double slot. s12 00:01:10.240 --> 00:01:15.760 And first of all, since I've got you here today, I should have people tell you about the history. s13 00:01:15.760 --> 00:01:21.759 You should tell people about the history of AI and your journey through it and how you ended up here. s14 00:01:21.759 --> 00:01:37.040 Um, so, uh, sit back, um, get ready for story time and we're going to have the really slow build and we're going to hear about all about the history of AI for 20 minutes and then we're going to hear the details s15 00:01:37.040 --> 00:01:39.759 about what Moon Lake is doing. s16 00:01:39.759 --> 00:01:42.880 Um, he looks very persuasive there, doesn't he? s17 00:01:42.880 --> 00:01:47.439 Whereas at the time we were having this conversation, I was having a very bad hair day. s18 00:01:47.439 --> 00:01:52.240 So I had very little um choice um to agree to this task. s19 00:01:52.240 --> 00:02:04.000 Um so here I am with some long-term remarks on how did I and all of us get to this moment from the far ago days um when AI didn't work. s20 00:02:04.000 --> 00:02:11.200 Um so the very beginning of AI was the Dartmouth summer research project in 1956. s21 00:02:11.200 --> 00:02:24.720 It was for the holding of this um kind of summer group project was when um John McCarthy coined the term AI and got together um this group of people. s22 00:02:24.720 --> 00:02:35.120 Um so that one on the back right there, that's John McCarthy, there's Minsky and both most of them look like a real bunch of geeks as you can see. s23 00:02:35.120 --> 00:02:43.840 Um but the attractive one on right on the right end is you know my personal hero as more of a language guy um Claude Shannon. s24 00:02:43.840 --> 00:02:48.239 So this is sort of the start of AI 1956. s25 00:02:48.239 --> 00:02:50.879 It's where the the term AI came from. s26 00:02:50.879 --> 00:02:56.239 But really there's other stuff that came earlier than 1956. s27 00:02:56.239 --> 00:03:01.519 So really starting in the 40s and early in the 50s there was work on cybernetics. s28 00:03:01.519 --> 00:03:08.800 So cybernetics sought to tie together communications control and feedback in living things and computers. s29 00:03:08.800 --> 00:03:18.640 Um so uh you know kubernetes is really the same word as cybernetics just less anglicized um but has very different meaning. s30 00:03:18.640 --> 00:03:25.120 Um yeah so this led to the earliest work on neural networks and the perceptron network. s31 00:03:25.120 --> 00:03:40.400 So here's Frank Rosenlat's perceptron um that was you know predated the term artificial intelligence and as you can see like in those days you actually used to wire neural networks now this modern matrix multiplication s32 00:03:40.400 --> 00:03:54.720 but another thing that actually started before the term artificial intelligence and has been my long-term area of interest is doing things in natural language processing and natural language processing s33 00:03:54.720 --> 00:03:57.439 began as machine translation. s34 00:03:57.439 --> 00:04:05.439 And so here we are again in 1954, two years before the term artificial intelligence was coined. s35 00:04:05.439 --> 00:04:08.959 And here's the front page of the New York Times. s36 00:04:08.959 --> 00:04:20.000 Now, there's a way in which the front page of the New York Times in 1954 is surprisingly reminiscent of the issues that you see in the front page of the New York Times s37 00:04:20.000 --> 00:04:25.040 um in 2026 because it's exactly the same issue. s38 00:04:25.040 --> 00:04:28.560 President proposing ending citizenship for people. s39 00:04:28.560 --> 00:04:33.360 Um not much has changed in the um intervening 75 years. s40 00:04:33.360 --> 00:04:37.919 Um allegations of various kinds of behavior akin to treason. s41 00:04:37.919 --> 00:04:39.600 All sounds very familiar. s42 00:04:39.600 --> 00:04:41.840 Um, but this was the top half of the page. s43 00:04:41.840 --> 00:04:46.880 And if you went down to the bottom half of the page, um, you found this article. s44 00:04:46.880 --> 00:04:51.520 Russian is turned into English by a fast electronic, um, translator. s45 00:04:51.520 --> 00:05:02.320 A public demonstration of what's believed to be the first successful use of a machine to translate meaningful text from one language to another took place here yesterday afternoon. s46 00:05:02.320 --> 00:05:07.520 Um, and here's a little bit of video of showing that system. s47 00:05:11.919 --> 00:05:30.479 into one of the first nonmerica were made that the computer would replace most human translators. s48 00:05:32.560 --> 00:05:33.280 Okay. s49 00:05:33.280 --> 00:05:37.199 Um, and we'll come back to that again in a little bit. s50 00:05:37.199 --> 00:05:53.680 Um, but coming off of the founding of AI, um, by John McCarthy, fairly soon after that, John McCarthy moved to Stanford and founded Stanford Artificial Intelligence, the Stanford Artificial Intelligence Lab, s51 00:05:53.680 --> 00:05:56.000 starting from 1963. s52 00:05:56.000 --> 00:06:01.840 And the original Stanford AI lab was in this um, building up in the foothills. s53 00:06:01.840 --> 00:06:13.360 Um if you down if you know the South Bay well um if you've ever been to a restraero and you look over next to a restraero um where there's the Portola pastures s54 00:06:13.360 --> 00:06:25.120 um horse area or if you ride a horse um that's where this AI lab used to be and that was the site of many of the um founding um work in artificial intelligence s55 00:06:25.120 --> 00:06:26.960 and also a lot of other stuff. s56 00:06:26.960 --> 00:06:35.360 Um the Stanford AI lab was the where the very first video game tournament was played in 1972. s57 00:06:35.520 --> 00:06:52.240 Um now McCarthy himself um was um mathematical logician and nearly all of his work was sort of building out these not notions of mathematical logic but in his thinking he was very wide ranging s58 00:06:52.240 --> 00:07:09.199 and so he he liked the idea of how could we build an embodied artificial general intelligence and was very happy to sort of provide the facilities and the money for other people to start to explore this. s59 00:07:09.199 --> 00:07:24.400 Um so this was the Stanford AI Labs um very first robot um the Stanford cart not very fancy um the old um sale building um but more relevant to what we're going to talk about today s60 00:07:24.400 --> 00:07:33.520 for simulation infrastructures for AI um was another robot um Stanford SRRI shaky robot s61 00:07:33.520 --> 00:07:52.560 mach audio Um, so this was a robot that could um perceive the environment, move around, move boxes and so on in the environment and do things like that. s62 00:07:52.880 --> 00:07:53.520 Okay. s63 00:07:53.520 --> 00:08:07.520 And so this sort of started to explore this idea of having an embodied in artificial intelligence that had an understanding of a world and its environment. s64 00:08:07.599 --> 00:08:11.919 Um, just one little distraction that's not really about AI. s65 00:08:11.919 --> 00:08:16.080 Um, who here has a connection to Stanford? s66 00:08:16.080 --> 00:08:18.000 Stanford connections. s67 00:08:18.000 --> 00:08:20.479 Who here has a connection to Berkeley? s68 00:08:20.479 --> 00:08:22.000 Berkeley connections. s69 00:08:22.000 --> 00:08:23.199 Less Berkeley people. s70 00:08:23.199 --> 00:08:25.120 Anybody with a connection to both places? s71 00:08:25.120 --> 00:08:27.440 There are some people who've been to both of them. s72 00:08:27.440 --> 00:08:28.800 No. Okay. s73 00:08:28.800 --> 00:08:34.479 Um, so I just have to put in my advertisement for early Stanford. s74 00:08:34.479 --> 00:08:42.959 Um if you're an artificial intelligence person you know the story in the early days is all Stanford. s75 00:08:42.959 --> 00:08:53.839 I mean there you know there are two different stories I can tell right you know if you're in the history of west coast universities really in the first half of the 20th century s76 00:08:53.839 --> 00:09:08.160 all the prominent stuff was Berkeley and that Stanford was um this sort of pokey regional school but once you get to the second half of the 20th century which is the era of computers and AI s77 00:09:08.160 --> 00:09:20.480 um it all happened at Stanford um So that you know if you look at things like the beginnings of the internet, the Arpanet, um here it is in 1972 s78 00:09:20.480 --> 00:09:32.000 and it connects up some UC campuses, Berke um UCLA and Santa Barbara and connects to Stanford, MIT, Harvard, but no Berkeley. s79 00:09:32.000 --> 00:09:33.920 Go ahead to 1980. s80 00:09:33.920 --> 00:09:36.800 Um Berkeley still isn't part of the internet. s81 00:09:36.800 --> 00:09:42.160 you know, it's made it as far as Hawaii and London and Berkeley is still not there. s82 00:09:42.160 --> 00:09:48.320 Um, and really in all of this period, there just wasn't AI at Berkeley. s83 00:09:48.320 --> 00:09:51.600 I mean, there was other stuff that went on at Berkeley, I should be fair. s84 00:09:51.600 --> 00:09:53.600 There was important systems work. s85 00:09:53.600 --> 00:10:00.320 There was ingress database and BSD Unix, which some of you probably remember if you have gray hair like me. s86 00:10:00.320 --> 00:10:09.279 Um, but you know, essentially through the 60s and 70s and the first half of the 80s, there was just no AI at Berkeley. s87 00:10:09.279 --> 00:10:25.360 And so AI at Berkeley really only got underway in 1986 um when two fresh Stanford PhD grads um Jendra Malik and Stuart Russell um moved from Stanford um to Berkeley to take up professorships. s88 00:10:25.360 --> 00:10:27.279 Now of course that's 40 years ago now. s89 00:10:27.279 --> 00:10:33.040 So, they've had they've had um AI for a while at Berkeley now, but not in the old days. s90 00:10:33.040 --> 00:10:44.399 Um, if you'd like to know more about this history of the old days, um, a couple of years ago, we actually put together, um, a story of the first 60 years of, um, AI at Stanford. s91 00:10:44.399 --> 00:10:52.160 Conveniently, um, the 60 years stopped just before the arrival of chat GPT and large language models. s92 00:10:52.160 --> 00:10:56.640 Um, and you can, um, find it on YouTube. s93 00:10:56.640 --> 00:10:57.760 Um, okay. s94 00:10:57.760 --> 00:11:02.320 So, that was my very brief potted history of AI. s95 00:11:02.320 --> 00:11:05.120 Um, what did I do for my life? s96 00:11:05.120 --> 00:11:10.720 Well, what I did for my life was to be a natural language processing person. s97 00:11:10.720 --> 00:11:17.120 And so, a natural language processing that was where we developed language models. s98 00:11:17.120 --> 00:11:25.120 And language models actually in some form go back a very very long way. s99 00:11:25.120 --> 00:11:32.560 So the first language model was proposed by Andre Marov who invented Markoff models right. s100 00:11:32.560 --> 00:11:46.480 So in developing the idea of Markoff models he actually did it with language um taking a novel of Pushkin's Eugene Anagen and developed a character level language model over that. s101 00:11:46.480 --> 00:12:01.120 Um but the famous thing is then back to Claude Shannon who we saw earlier who sort of formalized information theory and started to build um word and character engram language models in the late 1940s s102 00:12:01.120 --> 00:12:14.800 which is the dominant stuff that we went on and using and in particular um then in 1975 again so back in the sort of early part of AI um a famous group at IBM s103 00:12:14.800 --> 00:12:27.680 Fred Gelan's group at IBM sort of defined well they came up with the term language model that we still use today and they defined this idea of probabilistic models of text s104 00:12:27.680 --> 00:12:34.480 that could be used as a basis for all kinds of speech and natural language processing applications. s105 00:12:34.480 --> 00:12:49.920 So really from very early on in the speech and NLP tradition, language models were seen as a central technology that other people in other areas of AI and um machine learning just didn't really know about. s106 00:12:49.920 --> 00:13:00.320 And it was what enabled um interesting good things to happen early in speech and various areas of NLP. s107 00:13:00.320 --> 00:13:11.360 So both early speech recognition systems but also the kind of um machine translation that you got at Google starting about 2007. s108 00:13:11.360 --> 00:13:14.880 It was considerably powered by language models. s109 00:13:14.880 --> 00:13:29.760 And so I think there's actually a kind of an interesting story here of the development of our modern AI um sorry which is um slightly different to the story most people remember s110 00:13:29.760 --> 00:13:43.680 which um is a story that's dominated by the vision story because the vision sort of came in most people's head to be seen as the sort of entry place of neural networks in a large scale. s111 00:13:43.680 --> 00:13:48.320 Um but nevertheless I'll point out the limitation of this early work in language models. s112 00:13:48.320 --> 00:13:52.880 It was used for spelling correction, machine translation, all of these things. s113 00:13:52.880 --> 00:13:59.279 But at this time, nobody thought of language models as these were going to solve artificial intelligence. s114 00:13:59.279 --> 00:14:11.199 We still all bought the old AI story of that we're going to need memories, knowledge representations, planning systems, reasoning systems, all of these classic AI things. s115 00:14:11.199 --> 00:14:11.839 Yeah. s116 00:14:11.839 --> 00:14:18.000 So here's the sort of history of large language models from an NLP suspect perspective. s117 00:14:18.000 --> 00:14:24.639 So the very first mention of the term large language models was in 1998 as far as I can tell. s118 00:14:24.639 --> 00:14:30.399 But you know really that was sort of a how are we going to store all this text um to build a model. s119 00:14:30.399 --> 00:14:40.800 So the first interesting um connection of large language models was in 2000 when Joshua Benjio and colleagues defined new probabilistic language models. s120 00:14:40.800 --> 00:14:43.360 the first neural language model. s121 00:14:43.360 --> 00:14:57.600 Um, but as you can see from those stats of 32 million um token corpus, 31,000word vocabulary, this was a teeny model because at that point they just didn't have enough compute to do anything interesting. s122 00:14:57.600 --> 00:15:08.639 Um, but interestingly as early as 2007 at Google they were able to solve the compute problem and the data problem. s123 00:15:08.639 --> 00:15:19.839 Um, so most people forget this now, but in 2007, um, Google had a language model that was built on two trillion tokens of text. s124 00:15:19.839 --> 00:15:29.040 So, you know, that's a bit smaller than the state-of-the-art models now that might be trained on 15 trillions of text, but it's actually the same order of magnitude, right? s125 00:15:29.040 --> 00:15:38.880 So we all already in sort of two decades ago there was the scale of data and compute to build a form of language model. s126 00:15:38.880 --> 00:15:48.880 The big problem was they didn't have enough model flexibility that the kind of modern powerful neural networks that we use today hadn't yet been invented. s127 00:15:48.880 --> 00:15:58.079 Um and so it was then only in 2018 that modern large language models started to appear using transformers. s128 00:15:58.079 --> 00:16:03.199 But the early versions of those um were back to small amounts of data. s129 00:16:03.199 --> 00:16:08.000 So the early GPT model only used 3.3 billion tokens. s130 00:16:08.000 --> 00:16:11.199 So down three orders of magnitude in size again. s131 00:16:11.199 --> 00:16:13.759 So we reverted to not enough data. s132 00:16:13.759 --> 00:16:26.399 And then it was eventually only in 2020 with GPT3 forward that all of those things came together and language models really took off as we know about it now. s133 00:16:26.399 --> 00:16:31.920 Um and so that led to this surprising victory of natural language processing. s134 00:16:31.920 --> 00:16:44.880 I mean it actually gives me a bit of a laugh um that if in the sort of mass media if you see a reference to AI these days with pretty high probability they're actually going to be talking about a large language model s135 00:16:44.880 --> 00:16:50.880 which wasn't the way it used to be where NLP used to be a fairly marginal area of artificial intelligence. s136 00:16:50.880 --> 00:16:57.759 And as we all know, these large language models have allowed us to do amazing amazing things. s137 00:16:57.759 --> 00:17:07.439 And so the ability of these systems to reason and solve complex problems like math problems is just completely stunning. s138 00:17:07.439 --> 00:17:22.880 I mean, you know, even for someone like me who's spent my 30 years working in natural language processing, I kind of find it hard to believe that you can pick really difficult math problems of a kind I certainly could not solve myself s139 00:17:22.880 --> 00:17:31.760 and just feed it into a large language model and somehow it can um chunk along doing its test time thinking and come up with right answers. s140 00:17:31.760 --> 00:17:37.600 I mean it's just been a sort of a stunning breakthrough from an unexpected direction. s141 00:17:37.600 --> 00:17:49.440 Um so that's been most of my world but then here we are um now getting back into the main topic of the talk of well where are we um s142 00:17:49.440 --> 00:18:06.080 why is embodied artificial general intelligence a north star that we should be looking at and the reason for that is even though it's been so amazing what can be done with large language models s143 00:18:06.080 --> 00:18:10.640 they're still this textbased description of the world. s144 00:18:10.640 --> 00:18:16.640 And it turns out that you can do a lot of stuff with a textbased description of the world. s145 00:18:16.640 --> 00:18:26.160 I think it's true that we can do just way more than almost anybody believed possible with a textbased description of the world. s146 00:18:26.160 --> 00:18:35.120 Certainly a lot of prominent people in robotics and computer vision spent a decade saying, "Oh, you'll never be able to do that with a large language model." s147 00:18:35.120 --> 00:18:39.039 where actually we've been able to do a lot of that with a large language model. s148 00:18:39.039 --> 00:18:49.520 But still at the end of the day um we do actually want to deal with the world around us and have artificial intelligence that can operate in our world. s149 00:18:49.520 --> 00:18:55.440 And so the question is then how can we build these embodied artificial general intelligences? s150 00:18:55.440 --> 00:19:08.400 And so gradually I started to um get a bit more interested in well how can we start to incorporate the visual world and I started to look at things like visual question answering s151 00:19:08.400 --> 00:19:17.039 and how you could connect between um text and then generative models of visual worlds and reason about that. s152 00:19:17.039 --> 00:19:28.000 And so today I'm going to be telling you more about how you can build out that line of work and be using simulation as a way to start to approach s153 00:19:28.000 --> 00:19:31.280 um embodied artificial general intelligence. s154 00:19:31.280 --> 00:19:34.320 So why do we want simulation? s155 00:19:34.320 --> 00:19:42.720 Um there are sort of two ways that you can go about um starting to build intelligent models of the physical world. s156 00:19:42.720 --> 00:19:49.679 One way of doing it is that you're actually going to directly learn in the real world. s157 00:19:49.679 --> 00:20:00.960 So you can um set up hardware or collect your YouTube videos in the real world and start learning intelligent policies for how to act. s158 00:20:00.960 --> 00:20:10.000 So this was the kind of approach that was used in Google X's QOP um work that you've got your um rows of robot arms that are doing things. s159 00:20:10.000 --> 00:20:16.720 you're recording what they're doing and you're in the physical world starting to learn a policy. s160 00:20:16.720 --> 00:20:22.160 Um, that's a really unpleasant way to try and make progress. s161 00:20:22.160 --> 00:20:35.200 So, when people are do making progress in this kind of world, you're getting about 10,000 hours of teleyop, that means a human is moving the robot around um data s162 00:20:35.200 --> 00:20:37.679 to start to train a model. s163 00:20:37.679 --> 00:20:40.320 um it's not a very appealing picture. s164 00:20:40.320 --> 00:20:50.880 If we want to start having generally good robotics, we're just not going to get very far if we're sort of um chugging along at the speed of these robots s165 00:20:50.880 --> 00:20:52.880 um moving. s166 00:20:52.880 --> 00:21:00.240 But interestingly, if we go back to Shaky in the 1970s, Shaky had a different answer. s167 00:21:00.240 --> 00:21:08.080 Shaky said, "Well, we shouldn't just have the real world that's surrounding us. s168 00:21:08.080 --> 00:21:19.679 We should also have in the robot's head a world model that has an internal representation of what the world is like. s169 00:21:19.679 --> 00:21:31.360 So this idea of a world model as an abstracted internal representation of the world which can be used for simulation and planning that's an idea that goes back to cognitive science. s170 00:21:31.360 --> 00:21:36.559 So it was first proposed by Kenneth Craig um in the 1940s. s171 00:21:36.559 --> 00:21:44.000 So he argued that humans and other creatures have world models inside their own heads. s172 00:21:44.000 --> 00:21:56.559 Um so um that they can think about alternatives and how they're likely to play to play out and therefore they can think of good plans um before acting. s173 00:21:56.559 --> 00:22:03.760 And so that's also what we'd like our robots to do is to have the same kind of abilities. s174 00:22:03.760 --> 00:22:06.159 Um so this was Shaky's world model. s175 00:22:06.159 --> 00:22:18.480 So, Shiki had a kind of a blocks world where it was actually a grid world as you might remember from your early AI textbooks, but it was representing um what was in different places in the world s176 00:22:18.480 --> 00:22:26.720 and what were the attributes of different things and it was all stored in this kind of logical representation in those days. s177 00:22:26.720 --> 00:22:37.039 So how now can we start um building a 21st century version of an embodied artificial general intelligence? s178 00:22:37.039 --> 00:22:44.159 And I think the right way to do it is to work out how to build good action condition world models. s179 00:22:44.159 --> 00:22:58.159 And today I want to talk about a practical effective way to build a simulation infrastructure that can be used um for as a basis from embodied AGI. s180 00:22:58.159 --> 00:23:10.640 So the idea of a world model is now normally formalized in terms of reinforcement learning ideas that what we do is we have observations of a world but we assume that underlying s181 00:23:10.640 --> 00:23:18.080 those we have a semantic abstracted representation of the state of the world in our head. s182 00:23:18.080 --> 00:23:30.320 And what the world model does is gives us an ability to try and predict um when an action is taken in one state, what new state is going to emerge. s183 00:23:30.320 --> 00:23:39.120 And so the crucial thing there is that the world model is abstracted and has more semantics. s184 00:23:39.120 --> 00:23:51.840 Quite a lot of the time when people have talked about world models, they haven't really been thinking about the abstraction and the semantics, they've just been talking about can you produce s185 00:23:51.840 --> 00:23:54.320 beautiful generative AI video. s186 00:23:54.320 --> 00:24:05.679 Um so here's um Genie 3. Um this is kind of a cute example by Riley Goodside who was the same person who got a lot of fame in the early LLM s187 00:24:05.679 --> 00:24:08.320 days by being a good prompt engineer. s188 00:24:08.320 --> 00:24:18.400 um and he's now playing around um here with Genie 3. And you know the the kind of things you can do with Genie 3, you know, it looks beautiful, s189 00:24:18.400 --> 00:24:30.960 it's wonderful, it's pretty fluid, it seems great, but you know, most of visual AI for this period has been just judged by the pixels. s190 00:24:30.960 --> 00:24:39.840 If you've got beautiful pixels, you've got a beautiful piece of software, but these pixels are trying to simulate observations. s191 00:24:39.840 --> 00:24:45.200 They don't actually have or represent any of the semantics behind the world. s192 00:24:45.200 --> 00:24:48.640 And therefore, they aren't good at realism. s193 00:24:48.640 --> 00:24:51.600 They aren't good for giving a basis to plan. s194 00:24:51.600 --> 00:24:57.200 They aren't good for actually having a robust simulation infrastructure of the real world. s195 00:24:57.200 --> 00:25:12.480 And so we want this kind of simulation infrastructure because with a good simulator we actually have causal knowledge of how the world works which allows us to predict and plan how things will work in any situation. s196 00:25:12.480 --> 00:25:20.880 And so that's the kind of world um that we're wanting to have and make available at Moon Lake. s197 00:25:20.880 --> 00:25:25.200 So the starting point is a real world observation. s198 00:25:25.200 --> 00:25:39.279 Um, so given an image or a bit of video, we want to be able to interpret this image as an observation which is always partial of what's actually in the underlying world. s199 00:25:39.279 --> 00:25:49.919 And then what we want to do is reconstruct a model of this world which actually allows us to do stuff in it and see how it reacts. s200 00:25:49.919 --> 00:26:02.880 that this will give us a basis of being able to work out causality, work out how to plan and reason in a repres representation condition way which gives us the basis s201 00:26:02.880 --> 00:26:08.320 for intelligent robot actions inside this world. s202 00:26:08.320 --> 00:26:10.880 So what might one do here? s203 00:26:10.880 --> 00:26:23.919 Well, the first thing you can do is take that um picture and feed it into one of our well-known um other companies products and say, "Okay, make a simulation of this world." s204 00:26:23.919 --> 00:26:27.520 Um and so this is what you get from Marble. s205 00:26:27.520 --> 00:26:30.400 And it's a pretty good simulation of the world. s206 00:26:30.400 --> 00:26:40.640 And if what you want to do is just uh walk around in this world and see it from different angles, this works pretty well. s207 00:26:40.640 --> 00:26:53.840 Um the question is is this a good simulation and whether it's a good simulation depends on what problem you want to solve in the real world. s208 00:26:53.840 --> 00:27:00.400 And then is does this give you sufficient information to solve the problem in the real world? s209 00:27:00.400 --> 00:27:08.799 And if all you want to do in the real world is to be able to wander around and see the view from different angles, then this is great. s210 00:27:08.799 --> 00:27:10.000 We're done. s211 00:27:10.000 --> 00:27:21.120 But a lot of the time what you'd like to do in the real world is understand the objects are here and to be able to do things with them. s212 00:27:21.120 --> 00:27:21.440 Right? s213 00:27:21.440 --> 00:27:22.960 So there's some objects here. s214 00:27:22.960 --> 00:27:26.640 There's a kettle and there's a box and there's a cup. s215 00:27:26.640 --> 00:27:32.080 But in the marble world, there's nothing you can actually do with these things. s216 00:27:32.080 --> 00:27:36.720 All you can do is sort of wander around as a disembodied figure. s217 00:27:36.720 --> 00:27:49.440 Um, and so for a lot of purposes such as doing things with robotics or other physically accurate worlds, um, we need to be able to have more understanding in our model of the world, s218 00:27:49.440 --> 00:27:52.400 a better, more detailed simulation. s219 00:27:52.400 --> 00:28:02.080 And so at Moon Lake, we're wanting to work out what people actually want to do with their simulation, and then to build the kind of simulation that will power that. s220 00:28:02.080 --> 00:28:07.360 So you might think, oh, I actually want to be able to move around the objects in the world. s221 00:28:07.360 --> 00:28:14.000 And so that's the kind of starting point of the kind of thing that we're trying to do at Moon Lake. s222 00:28:14.000 --> 00:28:32.640 So if we want to um have a world in which we can move things around then we're saying okay kind of like um image blaster a system like that we actually need to take this world and understand what's in it and then have objects that can be manipulated. s223 00:28:32.640 --> 00:28:46.320 So we separate out a background and then objects inside that background with then being able to sort of have the background world in a representation that's similar to marble. s224 00:28:46.320 --> 00:28:52.080 But then in the foreground there are various kinds of objects that we can manipulate and move around. s225 00:28:52.080 --> 00:29:01.760 Now, well, that's a start, but um in the previous picture, um the tea box was always closed. s226 00:29:01.760 --> 00:29:07.520 And we might wonder if you can open up the tea box and see what's inside it. s227 00:29:07.520 --> 00:29:14.320 Um and well, this is sort of a part of how observations of a world are always partial. s228 00:29:14.320 --> 00:29:18.799 Um for what we could see in the actual photo, there was a closed tea box. s229 00:29:18.799 --> 00:29:21.520 We couldn't even see the tea box very well. s230 00:29:21.520 --> 00:29:25.360 How could we possibly know what's inside it? s231 00:29:25.360 --> 00:29:34.080 And well, the answer to the way we can know what's inside it in the 21st century is we go off and do a little bit re of research. s232 00:29:34.080 --> 00:29:47.039 We fire up um getting information um off the web um in a a rag style fashion um to work out what's there. s233 00:29:47.039 --> 00:29:54.480 And then we can find images of this um tea um box on the web which show it in more detail. s234 00:29:54.480 --> 00:29:57.200 They show a picture of what it's like when it's open. s235 00:29:57.200 --> 00:29:58.640 There are descriptions of it. s236 00:29:58.640 --> 00:30:01.679 Organic luxury tea bag collection, leather gift box. s237 00:30:01.679 --> 00:30:03.039 It gives its size. s238 00:30:03.039 --> 00:30:04.640 We can find all about it. s239 00:30:04.640 --> 00:30:13.679 So therefore, we can build um in our simulation um a tea box with an understanding of the contents of that tea box. s240 00:30:13.679 --> 00:30:15.200 Well, that's really good. s241 00:30:15.200 --> 00:30:20.559 Um, so how now we have um the tea box with tea bags inside it. s242 00:30:20.559 --> 00:30:26.480 But if we do nothing else, um, they're just sort of sitting there and movable. s243 00:30:26.480 --> 00:30:34.880 So we'd like to realize the fact, well, wait a minute, tea bags you can lift up and you can take out of a tea box. s244 00:30:34.880 --> 00:30:46.000 So then we need to start having these teaags also um be objects that are modeled in our simulation that they have a size and an ability to be manipulated s245 00:30:46.000 --> 00:30:47.360 as well. s246 00:30:47.360 --> 00:30:51.200 Um so how are we doing all of this? s247 00:30:51.200 --> 00:31:05.840 And so a distinctive part of what's happening at Moon Lake is building although part of this is in the world of vision a lot of what we're doing is actually back in the world of code. s248 00:31:05.840 --> 00:31:18.159 So this is picking up on the idea um that what's normally referred to as language models but these days are really symbolic models which work on not only human languages s249 00:31:18.159 --> 00:31:27.360 but also math code and things like that that they that has been just a very powerful substrate with which to make progress. s250 00:31:27.360 --> 00:31:40.080 And so we are producing controllable manipulable world models by generating the controllable parts of these worlds by putting code under them. s251 00:31:40.080 --> 00:31:52.960 And in particular, um, we can train these code models using the same kind of loop engineering that many of you will have seen, um, with Claude that we're having a loop s252 00:31:52.960 --> 00:32:12.960 where we're writing code to render objects, um, and put on that um, diffusionbased textures, etc. We can then assess how good our render is against physical reality and then we can work out the deviations. s253 00:32:12.960 --> 00:32:19.279 We can revise the code and make better and better um renders of what goes along. s254 00:32:19.279 --> 00:32:30.960 And so that's the way that so we're building this detailed action conditional world model so that we can take actions on the objects in the world. s255 00:32:30.960 --> 00:32:45.120 Um so that means that we have a kind of neuros symbolic representation of the world here and neuros symbolic representations have a big advantage that the symbolic representation can be easily s256 00:32:45.120 --> 00:32:51.039 interfaced controlled edited and maintained in our human world. s257 00:32:51.039 --> 00:33:05.440 Um, and so the message I'd like to give as a little delta of a message here, um, is most of you are probably familiar with Mark Andre's famous statement 15 years ago, s258 00:33:05.440 --> 00:33:07.519 software will eat the world. s259 00:33:07.519 --> 00:33:12.000 And you know, that was mostly right. s260 00:33:12.000 --> 00:33:20.960 Um but I think it wasn't completely right because it's just not the case that software ate the physical world. s261 00:33:20.960 --> 00:33:32.640 Um it, you know, a lot of the world went virtual and um a lot of the world is inside computers now and yeah it could eat all of the processes s262 00:33:32.640 --> 00:33:41.360 of managing, counting, supplying all um records of suppliers, all the stuff that's in the virtual world. s263 00:33:41.360 --> 00:33:46.640 But we hadn't had the power for software to eat the physical world. s264 00:33:46.640 --> 00:34:02.080 Whereas the hope is with this new ability of being able to build verifiable simulations powered by code of the sort that I've just sketched for a moment there that this will allow us to actually have verifiable s265 00:34:02.080 --> 00:34:06.640 simulation which will also eat the physical world. s266 00:34:06.640 --> 00:34:08.560 And so that's the kind of thing we built. s267 00:34:08.560 --> 00:34:14.399 And we can go on from here and um you know keep on making further steps of this. s268 00:34:14.399 --> 00:34:14.639 Right? s269 00:34:14.639 --> 00:34:25.440 So we maybe don't want to only have tea bags um in foil containers, but we'd like to be able to um take them out of the foil container and we s270 00:34:25.440 --> 00:34:32.480 don't we somehow want to actually get the tea bag inside the cup because that's a useful step for making tea. s271 00:34:32.480 --> 00:34:42.480 And then of course we also want to have um the jug of um water which we want to be able to boil and then be able to pour that into the tea. s272 00:34:42.480 --> 00:34:57.760 Um and once we have a good world simulator like that, the idea of this is that we are building in all of the parts of the simulated world which will allow s273 00:34:57.760 --> 00:35:01.440 effective transfer into the real world. s274 00:35:01.440 --> 00:35:08.560 And it's important to think about that as to sort of what's necessary for different kinds of transfer. s275 00:35:08.560 --> 00:35:29.040 And the argument um that I we'd like to make is that you know normally uh simulation isn't complete and accurate in every detail because there are sort of parts of the world that are important to you and parts of the world that aren't important to you. s276 00:35:29.040 --> 00:35:32.400 I mean, this is the same with human world models, right? s277 00:35:32.400 --> 00:35:39.280 That a lot of the world we're not actually modeling at any time, but we're modeling the bits of it that are important to us. s278 00:35:39.280 --> 00:35:51.280 And it's having that control of which things you need to model and use is what needed then to give you the actual simulations that will be allowed to be effectively used s279 00:35:51.280 --> 00:35:52.960 um in the real world. s280 00:35:52.960 --> 00:35:58.160 So, what kind of applications um is this going to allow us to do? s281 00:35:58.160 --> 00:36:14.720 So the hope is that um rather than having to collect 10,000 hours of data in by teley op in the real world, we can instead build accurate simulations which will allow transfer to real. s282 00:36:14.720 --> 00:36:18.480 So let me show you the kind of progress that we've been making on this. s283 00:36:18.480 --> 00:36:32.480 So this is um not highuting robotics um but is the kind of physical AI that you find everywhere in the real world that there are things happening in physical processes s284 00:36:32.480 --> 00:36:38.960 um which you'd like to be able to um understand and automate. s285 00:36:38.960 --> 00:36:53.280 And so in particular, um, if we're going to realize any of the dreams of bringing back America as a great manufacturing economy, it seems like we have to work out how to be able to automate much more in the physical world. s286 00:36:53.280 --> 00:37:12.880 Well, um, using the Moonlake AI technology, what we can then do is say, so from this short video, we can automatically produce an accurate simulation of that world, turning it into a 3D world with enough detail about how things move s287 00:37:12.880 --> 00:37:24.640 and what objects are in the world that we can start to build on this and use it as a basis of simulated data which transfers refers accurately to the real world. s288 00:37:24.640 --> 00:37:31.760 So we then have this model which allows us to get 10,000 hours of simulation um for free. s289 00:37:31.760 --> 00:37:46.079 And so on the basis of that um we can then train up a robotic system that can operate um to actually um explore what you can do in this world s290 00:37:46.079 --> 00:37:50.800 and learn the way to operate in the world so it works. s291 00:37:50.800 --> 00:37:57.599 and then we can have the effective and cheap training of robotic systems. s292 00:37:57.839 --> 00:37:58.560 Okay. s293 00:37:58.560 --> 00:38:00.240 Um so that's the story. s294 00:38:00.240 --> 00:38:02.701 Um thanks a lot everyone. s295 00:38:02.701 --> 00:38:04.701 [applause] s296 00:38:09.760 --> 00:38:13.839 And I do have time for questions I believe. s297 00:38:16.320 --> 00:38:18.320 Thank you very much for the conversation. s298 00:38:18.320 --> 00:38:20.079 Um, super deep. s299 00:38:20.079 --> 00:38:30.000 My first question is, would this be applied to something like the gaming industry and how could they use it for we're seeing GTA 6? s300 00:38:30.000 --> 00:38:32.000 Everyone is talking about this new game. s301 00:38:32.000 --> 00:38:40.400 Um, I could see a very application there where you could have endless interaction with the world model that they're building. s302 00:38:40.400 --> 00:38:44.880 U, have you been working with the gaming industry at all or this is more applied to robotics? s303 00:38:44.880 --> 00:38:47.760 Um, so yeah, absolutely. s304 00:38:47.760 --> 00:38:52.800 Um, another hugely good area for this is in the gaming industry. s305 00:38:52.800 --> 00:39:06.079 And you know, the real world is then kind of a virtual world, but you can have a simulation of your gaming world um, and then be building the same kind of embodied intelligence. s306 00:39:06.079 --> 00:39:10.000 And the gaming industry has some really appealing attributes. s307 00:39:10.000 --> 00:39:15.200 Well, you know, there are millions and millions of game players, so there's lots of easy data to collect. s308 00:39:15.200 --> 00:39:20.960 Um the people who are gaming have goals which are fairly clearly known. s309 00:39:20.960 --> 00:39:25.760 So you can sort of learn in a reinforcement learning loop good ways to act. s310 00:39:25.760 --> 00:39:33.839 And so absolutely one of the applications that Moon Lake has explored is in the gaming industry and there's likely to be more of that. s311 00:39:33.839 --> 00:39:42.880 Um but recently we've been actually particularly emphasizing physical infrastructure and doing simulations in the physical infrastructure world. s312 00:39:42.880 --> 00:39:45.280 Thank you so much. s313 00:39:55.680 --> 00:40:12.560 Uh so it's fascinating to me that that you're turning back to neurosy symbolic representations and um the part of the original kind of old school AI involved a lot of ontology development and knowledge representation and logic and representation of that. s314 00:40:12.560 --> 00:40:26.079 Do you see a role for uh deepening the neuros symbolic representation using kind of like what we how ontologies have been developed previously to model um a representation of the real world? s315 00:40:26.079 --> 00:40:33.119 Um and do you see that as being a kind of an area of expansion and development for the type of tools that you're working in? s316 00:40:33.119 --> 00:40:48.880 Yeah, I mean I don't think we're quite going to go back to old style ontologies and knowledge bases, but I mean effectively that is what we're doing, you know, apart from it's in new clothing s317 00:40:48.880 --> 00:41:04.000 of um having it being um code that I mean I I do you know this is an interesting space and we can talk about it in more general right there is you know there's sort a purely s318 00:41:04.000 --> 00:41:15.839 neural approach in which your latent representation of the world is purely neural and so that's the kind of thing that the jeoper architecture is after I mean in some ways that's a good pure s319 00:41:15.839 --> 00:41:29.599 neural approach but on the other hand that's very hard for humans to connect to in any way or for other applications to connect to in any way and so s320 00:41:29.599 --> 00:41:41.839 our bet is that the sort of practical ical way to have a physical AI simulation infrastructure for the foreseeable future is to base it on symbolic representations. s321 00:41:41.839 --> 00:41:54.480 Taking advantage of sort of the huge power of code that we see everywhere around us in the sort of um codeex cla code era that those kind of symbolic representations s322 00:41:54.480 --> 00:42:01.280 allow um neural eye systems to reason, plan and do all of these things excellently well. s323 00:42:01.280 --> 00:42:08.400 And so to some extent yes that will be bringing back notions like older fashion knowledge representation. s324 00:42:14.079 --> 00:42:16.640 Mic keeps on. s325 00:42:21.440 --> 00:42:21.839 Hello. s326 00:42:21.839 --> 00:42:22.400 Hello. s327 00:42:22.400 --> 00:42:23.280 Oh hi. s328 00:42:23.280 --> 00:42:36.319 Um so a lot of our existing um it seems like industrial robotics use cases or to sort of automate existing sort of human manual intensive processes. s329 00:42:36.319 --> 00:42:52.240 Um would investment in development of like world models would that allow us to go explore novel new uh processes or activities where they're currently not accessible by human labor s330 00:42:52.240 --> 00:42:58.319 or by even the kind of existing um industrial manufacturing processes that we have. s331 00:42:58.319 --> 00:43:00.079 I mean sure absolutely. s332 00:43:00.079 --> 00:43:00.400 Yeah. s333 00:43:00.400 --> 00:43:15.520 So for the example at the end I showed a very old school um conveyor belt system but I mean you know we're also in this world of amazing things happening in humanoid robotics s334 00:43:15.520 --> 00:43:31.200 and for any of the new forms of robotics automation other cases you could think about doing things in space well you'll have exactly the same problem that You want to s335 00:43:31.200 --> 00:43:45.520 train up AI agents to be able to act in different scenarios and it's extremely costly and difficult to do that training in the real world. s336 00:43:45.520 --> 00:43:53.200 Um and it's very hard when training in the real world to get out into the sort of tale of rare cases. s337 00:43:53.200 --> 00:43:56.880 I mean you've seen that um for autonomous driving, right? s338 00:43:56.880 --> 00:44:01.280 There's a reason why autonomous driving kind of took 20 years to arrive. s339 00:44:01.280 --> 00:44:16.319 Um from you know Stanley breakthroughs of yay autonomous car wins the race to actually having Whimos um widely deployed is because you have this enormous tail and the way to get out to that tail for all of these new applications s340 00:44:16.319 --> 00:44:23.599 with um humanoid robotics, space robotics, etc. is to have good simulation. s341 00:44:26.560 --> 00:44:28.480 Hey, uh thank you for your talk. s342 00:44:28.480 --> 00:44:30.000 I love the energy. s343 00:44:30.000 --> 00:44:33.599 Uh my question is you showed a video of water. s344 00:44:33.599 --> 00:44:47.520 Uh so is everything like learned from videos or is like physics ingrained in anything like for example does your model know any physical properties of water like viscosity, Bernali's laws or like for example friction. s345 00:44:47.520 --> 00:44:57.680 uh for some use cases like self-driving maybe those things are like not important right you just need to know the velocity and stuff but like for other use cases I would imagine these physical properties are important so s346 00:44:57.680 --> 00:45:13.359 and we absolutely know physics I mean this is the sense in which this neurosyolic approach is you know you call a conservative approach if you will you know that this is absolutely using physics engines s347 00:45:13.359 --> 00:45:20.079 and knowledge of physics um to in its generation and control of movement, right? s348 00:45:20.079 --> 00:45:34.560 That if you're what you know when you've just started off with an image of water or and you want to understand how that's behaves, you're using a physics model to predict how it's going to behave. s349 00:45:36.240 --> 00:45:38.240 Okay. s350 00:45:40.079 --> 00:45:46.720 Um I know is there time for Oh, there's someone else at the mic. s351 00:45:47.440 --> 00:45:47.920 Hi. s352 00:45:47.920 --> 00:45:48.240 Hey. s353 00:45:48.240 --> 00:45:51.520 Uh, Professor Manning, thank you for this presentation. s354 00:45:51.520 --> 00:45:54.960 Um, I have a question about seem to row gap. s355 00:45:54.960 --> 00:45:56.880 So, I mean, simulation is great. s356 00:45:56.880 --> 00:45:59.520 It's cheap, is scalable. s357 00:45:59.520 --> 00:46:06.751 Um do you have a suggestion for um how we close the simulation to real gaps s358 00:46:06.751 --> 00:46:07.040 [snorts] s359 00:46:07.040 --> 00:46:22.400 encounter you know like the like fix part that we are not able to accurately simulate and like mechanical tolerancing back latches like all those things that we are not able to s360 00:46:22.400 --> 00:46:24.960 put into the simulations. s361 00:46:24.960 --> 00:46:41.040 Um yeah so traditionally the problem has always been the simtoreal gap and the simtoreal gap seeming too large so that a lot of the training has had to happen in the real world s362 00:46:41.200 --> 00:46:55.680 and we believe that the answer to that is you know effectively claude code loop we're now in this world in which we can do neural optimization. s363 00:46:55.680 --> 00:47:12.000 So we can have the simulation compared to behavior in the real world with video and we can automatically learn to shrink the sim to real gap in a way that just wasn't possible s364 00:47:12.000 --> 00:47:20.079 when you had people trying to handr write a physics simulation of something. s365 00:47:22.240 --> 00:47:23.839 This this was a great discussion. s366 00:47:23.839 --> 00:47:24.720 Thank you. s367 00:47:24.720 --> 00:47:41.040 Um tangential to the question that was just asked, how do you see like what do you see the biggest challenges are to adding like the sociote techchnical layer, the pieces, the the processes, the people, the authorities that make decisions s368 00:47:41.040 --> 00:47:44.079 on the objects that you're looking to simulate. s369 00:47:44.079 --> 00:47:49.359 What are the biggest challenges to developing a system that has accurate representation? s370 00:47:49.359 --> 00:48:01.839 So for example, if we want to model uh EV tolls or um doing you know forecasting of energy at airports uh you know things like that not necessarily extending beyond s371 00:48:01.839 --> 00:48:06.400 the robotics into these other um use cases areas. s372 00:48:06.400 --> 00:48:08.400 Um yeah that's a great area. s373 00:48:08.400 --> 00:48:13.839 Um in all honesty it's not something we've been um really dealing with at Moon Lake. s374 00:48:13.839 --> 00:48:29.599 Um, you know, at the there are still some limitations, but you know, large language models with their slurping up of enormous amounts of human behavior data that they're actually s375 00:48:29.599 --> 00:48:42.079 getting better and better at being able to simulate how human beings, different kinds of human beings are going to behave and react in different circumstances. s376 00:48:42.079 --> 00:49:01.119 So I think we are starting to approach the point in which we can have fairly good human behavior simulators that are being powered by the knowledge of large language models and there are a couple of companies that are now starting to look at that. s377 00:49:02.800 --> 00:49:03.280 Cool. s378 00:49:03.280 --> 00:49:04.000 Last question. s379 00:49:04.000 --> 00:49:06.000 Yeah. s380 00:49:09.440 --> 00:49:11.463 person over here has one for a long time. s381 00:49:11.463 --> 00:49:13.463 [laughter] s382 00:49:16.400 --> 00:49:17.520 Thank you. s383 00:49:17.520 --> 00:49:29.040 Um so if you look at um analog chips for instance right the physics is understood but as you go higher up the chemical operations then the heat and those things s384 00:49:29.040 --> 00:49:30.720 are still people are working on it. s385 00:49:30.720 --> 00:49:35.440 So that's not what you call completely um understood. s386 00:49:35.440 --> 00:49:52.480 Now I'll give you another example of say I'm looking at kidney related uh literature and stuff and then liver related stuff these things it gives a pretty good answer the kidney stuff it gives answer but when you look at the relations between the two people have somewhere s387 00:49:52.480 --> 00:50:06.640 u doctors have made a link and that's why you are able to see my take is if we operated completely in the latent space and we did not know the connection connections at a human level. s388 00:50:06.640 --> 00:50:16.800 Can we explore in the latent space completely and be able to find hidden connections over there that could translate into the real world or am I thinking something crazy? s389 00:50:16.800 --> 00:50:18.880 Yeah, I mean absolutely. s390 00:50:18.880 --> 00:50:31.520 I mean to the extent that we have a pretty good simulation, we can hope to find surprising discoveries that turn out to be correct in the simulated world. s391 00:50:31.520 --> 00:50:36.079 I mean, you know, things go both ways, right? s392 00:50:36.079 --> 00:50:42.000 There are also likely to be things that turn out to be true in the real world that weren't in our simulation, right? s393 00:50:42.000 --> 00:50:46.880 There's this famous statement about all models are wrong, but some models are useful, right? s394 00:50:46.880 --> 00:50:57.520 And that so anybody's world model, regardless of whether it's the one we're building or the world model in a human head, right, they're not always right. s395 00:50:57.520 --> 00:51:01.760 Sometimes we think a person's going to react in one way and they react in another way. s396 00:51:01.760 --> 00:51:13.040 But nevertheless, a lot of the time it can let us explore much more widely and discover new facts and new connections that we weren't aware of in the real world. s397 00:51:13.680 --> 00:51:15.680 Okay, s398 00:51:16.160 --> 00:51:16.720 awesome. s399 00:51:16.720 --> 00:51:17.839 That's a wrap everyone. s400 00:51:17.839 --> 00:51:20.480 Thank you so much, Chris.