WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=Sjfz1TqxzEs NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/Sjfz1TqxzEs.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:01.309 --> 00:00:03.309 [music] s2 00:00:12.360 --> 00:00:13.760 Thanks for the kind introduction. s3 00:00:13.760 --> 00:00:16.360 So, I'm Jason, co-founder of Diana. s4 00:00:16.360 --> 00:00:23.720 Today, I'll talk about how we're developing high-performance and very robust generalist robotics policies. s5 00:00:23.720 --> 00:00:24.600 Yeah. s6 00:00:24.600 --> 00:00:26.320 So, let's jump into it. s7 00:00:26.320 --> 00:00:35.400 So, today's talk will focus on, you know, how we're bringing robots into commercial grade and what we are doing to make these models, like I said, very high-performance and robust. s8 00:00:35.400 --> 00:00:37.480 So, just a brief introduction to our company. s9 00:00:37.480 --> 00:00:42.120 So, our mission is to build very robust foundation model autonomy in the real world. s10 00:00:42.120 --> 00:00:53.120 So, we want to train models and these days also hardware to have a single platform that can do many economically useful physical tasks in the real world, like the ones we're showing here. s11 00:00:53.120 --> 00:00:56.520 So, the company the company was founded in September 2024. s12 00:00:56.520 --> 00:01:10.360 We are a series A company, have raised about $120 million, and we And our thesis is that to actually bring robots into the real world, being at commercial grade doing useful tasks, s13 00:01:10.360 --> 00:01:19.800 the company needs to combine doing frontier research with a lot of commercial deployments, so we can build what we consider a research and deployment flywheel. s14 00:01:19.800 --> 00:01:28.520 Uh which is that the research and hardware we do in-house, the R&D informs the kind of tasks, the kind of workflows in the real world that we can commercialize. s15 00:01:28.520 --> 00:01:30.440 So, we actively try to deploy. s16 00:01:30.440 --> 00:01:36.600 So, these days we have more than five deployment sites doing a bunch of different tasks, which I will talk about in a bit. s17 00:01:36.600 --> 00:01:41.160 And then by building product and by deploying robots, it does several things. s18 00:01:41.160 --> 00:01:48.880 One is that it can help us gather high-quality deployment data, and it can also tell us what our models and what our hardware is not good at yet. s19 00:01:48.880 --> 00:01:54.960 So, it helps us sharpen our research focus to figure out what is the right problem to work on in robotics. s20 00:01:54.960 --> 00:01:58.600 Because if you're familiar with the robotics field, there are too many problems. s21 00:01:58.600 --> 00:02:01.920 There are too many different fields you can spend your effort on. s22 00:02:01.920 --> 00:02:11.480 And by deploying and by building a product, we know exactly the right kind of problems that we need to focus to actually make a robotics not just a demo or videos you see on YouTube, s23 00:02:11.480 --> 00:02:16.040 but rather in the real world impacting millions of people's life. s24 00:02:16.040 --> 00:02:25.600 So, before I jump into what we specifically work on at covariant, just a very quick introduction on the kind of models we're training for robotic manipulation. s25 00:02:25.600 --> 00:02:30.400 So, at a high level, uh we do data collection where you know, there's a lot of data being collected. s26 00:02:30.400 --> 00:02:36.200 So, here's a video of me uh manipulating a robotic hardware to, you know, fold a t-shirt. s27 00:02:36.200 --> 00:02:42.560 And once you collect enough of this kind of data, you can put all of them in a large neural network. s28 00:02:42.560 --> 00:02:59.840 And a neural network essentially takes in a, you know, representation of the world, in this case just camera feed of what the, you know, a table looks like, and the outputs robot's, you know, joint positions or torque to actually control the robot to do the task that you have collected data on. s29 00:02:59.840 --> 00:03:00.959 Right? s30 00:03:00.959 --> 00:03:07.239 So, you know, this is the high level of how you robot models in the real world function. s31 00:03:07.239 --> 00:03:14.000 So, how are we actually scaling this up to actually create models that's generalizable and it can do a lot of different tasks. s32 00:03:14.000 --> 00:03:23.800 So, at covariant we focus on what we call the pre-training data pyramid for the real world, where we gather a lot of diverse off-robot data, just because robotics data is very scarce. s33 00:03:23.800 --> 00:03:32.120 So, we have data captured from humans wearing cameras, uh from public data sets, and these days we're also considering some simulation data. s34 00:03:32.120 --> 00:03:37.680 But if you only have off-robot data, it's not actually enough to get robots to do very precise actions. s35 00:03:37.680 --> 00:03:50.120 So, on the robot themselves, we also collect diverse tasks collected in many different scenarios, on many different types of tasks, like industrial tasks, household, uh in the laundromat, and uh you know, hotels. s36 00:03:50.120 --> 00:04:00.080 And then finally, because we're also deploying robots, we can collect very high-quality deployment data, which helps the model to close train and test distribution gap. s37 00:04:00.080 --> 00:04:05.720 Because when you're developing robots, you know, most of the time your robots are in your facility, in your laboratories. s38 00:04:05.720 --> 00:04:08.480 But if you're deploying robots, it's in a very different environment. s39 00:04:08.480 --> 00:04:15.840 So, we found that by combining these three data sources, we can actually train large-scale foundation models that work very well in the real world. s40 00:04:15.840 --> 00:04:20.640 And so far, we have more than 200,000 hours of data in our training pipeline. s41 00:04:20.640 --> 00:04:33.040 And the model architecture roughly follows a high-level reasoning model with a low-level world action model that can actually output dexterous actions at fine-grained high frequency, right? s42 00:04:33.040 --> 00:04:44.400 So, this is the kind of architecture we have because in the real world, if you think about a robot doing physical tasks, it needs to have a semantic understanding of the world, but also needs to understand s43 00:04:44.400 --> 00:04:52.480 physical interaction at a fine-grained level to be actually able to, you know, recover from mistakes and do very precise actions to complete the task. s44 00:04:52.480 --> 00:05:03.080 So, once you combine, you know, this architecture with a lot of data, what happens is that the model can be rapidly fine-tuned to do a bunch of different tasks in the real world. s45 00:05:03.080 --> 00:05:08.520 Yeah, so here are just a gallery of the kind of benchmark tasks we're doing in the office. s46 00:05:08.520 --> 00:05:19.040 So, it spans from, you know, like cleaning trash, opening a box, to, you know, things like folding towels, folding t-shirts, and also just a bunch of other tasks our researchers have thought about. s47 00:05:19.040 --> 00:05:29.640 And what's really interesting is that with a good pre-trained model, even without any, you know, for some of these tasks I'm showing you here, they're not in the pre-training data at all. s48 00:05:29.640 --> 00:05:37.720 But with a good pre-training, you can actually post-train the model to do these kind of highly dexterous tool use tasks with very little amount of data. s49 00:05:37.720 --> 00:05:44.640 So, both of the videos you're seeing here have only been trained on with less than 1 hour of task-specific data. s50 00:05:44.640 --> 00:05:50.200 But you can see that the robot can repetitively do these tasks over and over without failure. s51 00:05:50.200 --> 00:06:00.000 And I think this is a stepping stone towards actual commercial grade real world deployment because in the real world physical tasks need to be done by humans and by robots s52 00:06:00.000 --> 00:06:01.160 over and over. s53 00:06:01.160 --> 00:06:09.800 And if your models are not reliable enough, then yes, you can shoot these kind of pretty demos, but it's still very far away from actual deployment. s54 00:06:09.800 --> 00:06:22.760 So that actually brings us to what I consider the current status quo for training large-scale foundation models for manipulation, which is that the generous models that can do many many tasks like the ones I've been showing you. s55 00:06:22.760 --> 00:06:29.640 Even though they make pretty videos, but what I would tell you is that the success rate is actually not super high. s56 00:06:29.640 --> 00:06:34.880 You know, when these models are actually deploying, they're about 80 to 90% success rate. s57 00:06:34.880 --> 00:06:44.320 But if you're stuck at 80 to 90% success rate, then you know, the chance that you can do the same task 10 times in a row is actually less than 0.1%. s58 00:06:44.320 --> 00:06:54.800 And on on the flip side, in the history of robotics, we have had very specialized robots and a very specialized machine learning pipelines to do singular tasks like pick and place. s59 00:06:54.800 --> 00:07:03.440 But this kind of pipeline, which is what I consider specialist models, aren't aren't very scalable, meaning that you can't take the same pipeline to just do a new task s60 00:07:03.440 --> 00:07:05.800 and do an arbitrary task very fast. s61 00:07:05.800 --> 00:07:16.440 So our mission is to resolve the status quo and you know, this kind of like dichotomy by training general purpose models that can both do many many tasks and also be reliable enough for commercial s62 00:07:16.440 --> 00:07:18.120 deployments. s63 00:07:18.120 --> 00:07:19.640 So how do we actually do that? s64 00:07:19.640 --> 00:07:31.240 In today's talk, I'll briefly talk about some of our progress on mastering very complex tasks very reliably and also taking the same skills to be performant not only in the environments s65 00:07:31.240 --> 00:07:40.600 in the laboratory environments that we're training, but also being able to deploy to arbitrary customer sites without fine-tuning or without fine-tuning adaptation. s66 00:07:40.600 --> 00:07:42.600 Okay, so let me get to this. s67 00:07:42.600 --> 00:07:54.080 Right, So, the first question we want to answer is can we take, you know, the general pre-training and post-training recipe that we had, but then turn these models into models that can be almost close to 100% s68 00:07:54.080 --> 00:07:56.920 robust on uh any task. s69 00:07:56.920 --> 00:08:00.280 So, this is the first research result we published last year. s70 00:08:00.280 --> 00:08:02.800 Uh for detail, you can check out our blog post. s71 00:08:02.800 --> 00:08:11.320 But, at a high level, you know, if we want to make a model 100% robust on many, many tasks, I think it's very good to first narrow down on a commercial s72 00:08:11.320 --> 00:08:17.360 use case that's reasonably hard, which allows you to make research progress, but also has commercial value. s73 00:08:17.360 --> 00:08:29.960 So, when we first started the company uh last year, what we discovered is that if you go to any, you know, uh restaurant, you know, uh fancy restaurant or dim sum places in the US, you'll see nicely folded napkin on the table, right, for you, right? s74 00:08:29.960 --> 00:08:32.960 If you go to a Cheesecake Factory, you'll see those napkins. s75 00:08:32.960 --> 00:08:42.760 And what happens is that in the back office of the restaurant, there's usually uh workers or, you know, restaurant employees that's folding these napkins one by one by hand. s76 00:08:42.760 --> 00:08:50.120 And that's a very mundane process, and a lot of the customers we have talked to are looking into robots that can actually do the same task. s77 00:08:50.120 --> 00:08:55.120 So, this is one of the earliest case study we did on how to train models to be very robust. s78 00:08:55.120 --> 00:09:03.920 So, our research result is a model called the Dyna-1, which is a generalist robot foundation model that's fine-tuned to do napkin folding. s79 00:09:03.920 --> 00:09:09.240 They can actually achieve 99.4% success rate over a 24-hour span. s80 00:09:09.240 --> 00:09:12.960 So, here is a time lapse of the model, you know, doing the task. s81 00:09:12.960 --> 00:09:19.880 And uh you know, it's sped up about a thousand times, so you can see the clock in the back running very quickly to show the progress. s82 00:09:19.880 --> 00:09:25.280 And uh my favorite part of the video is when the clock, you know, hits about like right now, right? s83 00:09:25.280 --> 00:09:29.760 Like 7:00 a.m. in the morning, so the lights, you know, actually come out in the outside. s84 00:09:29.760 --> 00:09:38.840 So, the environments are actually, you know, shifting over time due to the lighting as the robot folds the napkin, but the model's robust enough, and it just keeps going. s85 00:09:38.840 --> 00:09:41.720 So, how do we actually get to a model that can do this? s86 00:09:41.720 --> 00:09:47.200 And uh first of all, you know, this uh video we put out is not a one-time occurrence. s87 00:09:47.200 --> 00:09:55.960 The model can do this many, many times uh in the office, you know, so these are four distinct trials of 24-hour runs. s88 00:09:55.960 --> 00:10:00.320 So, before I dive into the technical detail, just to highlight how difficult the task is. s89 00:10:00.320 --> 00:10:13.000 So, you see that in the napkin folding task, you start out with a stack of napkin on the side, and what the robot has to do is like be very precise about picking out exactly one napkin from the stack, and then fold it, and then put it into a bin. s90 00:10:13.000 --> 00:10:23.200 And then a lot of the failure cases just come from the fact that these, you know, parallel jaw grippers we're deploying, you know, the gripper may not be precise enough to be able to pick out exactly one napkin. s91 00:10:23.200 --> 00:10:29.880 And in these situations, the model has to learn how to recover from the mistakes of pulling out extra napkins. s92 00:10:29.880 --> 00:10:43.240 And then secondly, the in commercial environment, different from a lab demo where the researchers like me are thinking of task success, there's actually very well-defined success criteria for these tasks. s93 00:10:43.240 --> 00:10:55.160 So, on the right, you see the difference between what we consider a grade five fold, which is a fold quality that the restaurant would accept versus a grade three, which is something that's below the acceptance criteria. s94 00:10:55.160 --> 00:11:03.640 And what you see is a barely like 1-in difference in how, you know, low the you know, the second fold seam of the napkin is. s95 00:11:03.760 --> 00:11:06.440 So, how do we actually get to a model that works really well? s96 00:11:06.440 --> 00:11:16.160 So, our internal attempt was kind of like the pre-training many, many hours of data and then post-training recipe that I told you about in the beginning. s97 00:11:16.160 --> 00:11:19.280 And doing this roughly gets you about 80% success rate. s98 00:11:19.280 --> 00:11:30.120 And what happens is that the model, you know, is doing fine in the beginning, but as soon as it makes a mistake, it'll typically go out of distribution, get stuck, and unable to recover. s99 00:11:30.120 --> 00:11:38.480 So, we have to do something more than the standard pre-training and post-training idea that's very popular in robotics and also in other fields. s100 00:11:38.480 --> 00:11:45.960 So what we did is that we developed what we consider reward models for complex long horizon manipulation tasks. s101 00:11:45.960 --> 00:11:53.040 So these are models that can, you know, look at a robot video and accurately score its progress towards solving a task. s102 00:11:53.040 --> 00:11:55.520 All right, so let me just play these videos again. s103 00:11:55.520 --> 00:12:07.120 So here's the same model, you know, being able to score how well the robot is doing these long horizon complex tasks as it's, you know, going from you know, starting of the task to finish. s104 00:12:07.120 --> 00:12:11.280 So you see that as the robot's completing a task, it's able to go from zero to one. s105 00:12:11.280 --> 00:12:20.640 And if you squint at these videos enough, you also see that whenever the robot is actually making some mistakes, there will be like slight dips in the reward model. s106 00:12:20.640 --> 00:12:26.720 And that actually becomes a very important insight into how to make these models very robust. s107 00:12:26.720 --> 00:12:31.920 So here's what happens when you run such model during like a autonomous run out of the robot. s108 00:12:31.920 --> 00:12:39.720 So you see that when the robot is doing fine folding napkins, the progress estimation is roughly monotonic going up, right? s109 00:12:39.720 --> 00:12:41.600 Because the robots are not messing up. s110 00:12:41.600 --> 00:12:45.640 But what's really interesting is is that let me just for fast forward a bit. s111 00:12:45.640 --> 00:12:49.200 Whenever the model starts to make mistakes, so here it is. s112 00:12:49.200 --> 00:12:51.680 This is the ninth napkin is folding. s113 00:12:51.680 --> 00:13:01.000 So you see that when the model is like making mistake, that's when the progress estimation, you know, starts to like, you know, show non-monotonic sign indicating that the robot is messing up. s114 00:13:01.000 --> 00:13:01.280 Right? s115 00:13:01.280 --> 00:13:13.839 And this is very important because once your model is like good enough in the 90% uh range, then it's very inefficient for humans to manually oversee the robot to detect its failure and then try to recover. s116 00:13:13.839 --> 00:13:19.720 But once we have this reward model, we can actually do what I consider uh scalable supervision. s117 00:13:19.720 --> 00:13:24.640 So you can just have the robot trying to fold napkins and then run this reward model in the background. s118 00:13:24.640 --> 00:13:34.960 So whenever a model does make a mistake, uh we as researchers or operators can immediately know the kind of video case that the model is struggling on and then do very targeted s119 00:13:34.960 --> 00:13:40.560 data collection and error recovery data for the model, then we can fine-tune the model again. s120 00:13:40.560 --> 00:13:54.440 So, the overall pipeline looks like a human-in-the-loop active learning process where we can use the reward model to help us catch the the kind of mistake the model is bad at and then do targeted collection to make the model better and iterate. s121 00:13:54.440 --> 00:14:04.200 And what we found is that once you iterate on this uh couple cycles, then you start to get a model that's extremely robust and can recover from all kinds of errors s122 00:14:04.200 --> 00:14:08.000 and finally bringing us closer to a commercial grade robots. s123 00:14:08.000 --> 00:14:16.080 So, here is just uh some of the uh you know highlights, I guess, during the 24-hour trial. s124 00:14:16.080 --> 00:14:28.000 So, what you saw there was the robot accidentally picked out more than one napkins and the model is able to, you know, separate the napkins and uh you know, here it's kind of doing that and uh be able to s125 00:14:28.000 --> 00:14:29.760 continue progressing. s126 00:14:29.760 --> 00:14:36.120 And what we found very interesting is that uh napkin folding is a deformable object manipulation task, right? s127 00:14:36.120 --> 00:14:41.520 So, there is almost infinitely many possible states or configuration that a napkin can get to. s128 00:14:41.520 --> 00:14:45.760 So, it's impossible to exhaustively collect data for all the error cases. s129 00:14:45.760 --> 00:14:55.120 But once we had done the active learning many, many times, we saw the model able to generalize to new ways of recovery from the mistakes made and continue to make progress. s130 00:14:55.120 --> 00:15:01.880 And that contributed to its ability to be able to uh fold that uh 99.4% success rate. s131 00:15:01.880 --> 00:15:06.440 So, here's uh what I found the most impressive bit from the trial. s132 00:15:06.440 --> 00:15:10.839 So, typically, if you look at robot videos, you know, they only show you the successful cases. s133 00:15:10.839 --> 00:15:18.960 But here, I wanted to highlight that even where a model accidentally pulled the entire napkins stack over, it's able to demonstrate this kind of error recovery behavior. s134 00:15:18.960 --> 00:15:25.640 That was very surprising to us when we were developing the model and it contribute to how it's able to just continuously run, right? s135 00:15:25.640 --> 00:15:39.400 So, here he made a big mess, but he's able to just do all kind of like very impressive like pulling you know, stretching behavior to recover from his mistake and then continuously going. s136 00:15:39.400 --> 00:15:47.960 Yeah, so you know, after developing such technology, we actually are successful at deploying our models at many many restaurants in the US. s137 00:15:47.960 --> 00:15:56.240 So, here's a real restaurant deployment of our robot folding napkins for the customers in their back office. s138 00:15:56.240 --> 00:16:03.480 And in addition to folding napkin, because the recipe is quite generalizable, now we also have models doing bunch of commercial tasks. s139 00:16:03.480 --> 00:16:12.920 So, here's at a real laundromat in Sacramento, and this time we fine-tune our model to do towel folding for the customer. s140 00:16:12.920 --> 00:16:21.920 So, you can see you know, there's a restaurant I guess a laundromat worker coming here to fill the bin over and over, and the robot just kept folding stacks of s141 00:16:21.920 --> 00:16:25.840 towels to serve the customer. s142 00:16:27.000 --> 00:16:32.160 So, now let's talk about we have a recipe to master very complex tasks. s143 00:16:32.160 --> 00:16:41.320 But, the caveat here is that in all the videos I've shown you so far, we have also collected data at the exact customer site where location that the robot is deployed. s144 00:16:41.320 --> 00:16:53.200 But, if you think about scaling robots to any task or to any customer site, then it'll be much better or more ideal if the models can readily generalize the environments they haven't seen before. s145 00:16:53.200 --> 00:17:09.040 So, this is what we have worked on in the I guess the end of last year where we figured out a data recipe to collect a lot of diverse tasks to allow the robots to be able to deploy at a new site without any additional data while maintaining the task performance. s146 00:17:09.040 --> 00:17:12.199 So, here's a demo we did at Coral 2025. s147 00:17:12.199 --> 00:17:17.959 So, Coral is the premier academic conference on robot learning, and it's held in Korea last year. s148 00:17:17.959 --> 00:17:24.160 So, you know, bringing a robot from the US to Korea was its whole challenge that I can talk about offline. s149 00:17:24.160 --> 00:17:33.840 But, the recipe we discovered was able to, you know, we just brought the robot to our, you know, exhibition booth and just dropped it there, and the robot can start folding s150 00:17:33.840 --> 00:17:35.760 many, many different t-shirts. s151 00:17:35.760 --> 00:17:39.920 And you see that the robot is facing the, you know, the conference attendees. s152 00:17:39.920 --> 00:17:50.280 So, many, many times there were people just deliberately trying to mess with the robot, use the t-shirt to cover up the robot's camera, then, you know, all kind of fancy stuff that you see at academic conferences. s153 00:17:50.280 --> 00:18:05.120 But, the model is able to continuously fold t-shirts over and over for 3 days straight at the conference just to demonstrate the ability to, you know, solve this task at environments it's never seen before, bringing it, you know, much closer to the kind of, you know, s154 00:18:05.120 --> 00:18:13.680 ideal, you know, go-to-market that you want to have for robotics company, which is just putting the robot to a site and it just starts working. s155 00:18:13.680 --> 00:18:17.080 And recently, we have also ventured into many different tasks. s156 00:18:17.080 --> 00:18:24.280 Like, we have a a partnership with Red Bull where we're opening Red Bull cans at, you know, Red Bull events. s157 00:18:24.280 --> 00:18:29.360 So, here's a video of our uh robot. s158 00:18:29.360 --> 00:18:32.480 Again, uh I guess this video won't play properly. s159 00:18:32.480 --> 00:18:34.080 So, but you get the point. s160 00:18:34.080 --> 00:18:45.520 We brought the robot to a Red Bull event and it's able to uh open uh these Red Bull drinks for, again, conference attendees over and over without failure, even though in this particular s161 00:18:45.520 --> 00:18:51.360 deployment, it's at a music festival, so the lighting's always changing in the background, but the model can continue. s162 00:18:51.360 --> 00:18:56.600 Yeah, but due to yeah, I guess the video will not play. s163 00:18:56.600 --> 00:19:09.960 So, just to summarize, uh our mission at AINA is to be able to build general-purpose models and robots that's both competent at many tasks, but also being able to focus and attain really high performance s164 00:19:09.960 --> 00:19:11.240 on commercial tasks. s165 00:19:11.240 --> 00:19:20.920 And I have demonstrated some of our recent progress on mastering complex tasks and also generalizing the skills to environments is the robots have never seen before. s166 00:19:20.920 --> 00:19:33.280 So, we are a series A stage company actively growing and hiring and if you're also interested in partnering with us for deployments and other things, feel free to reach out and then talk to me after the talk and thank you for listening. s167 00:19:33.280 --> 00:19:33.800 Awesome. s168 00:19:33.800 --> 00:19:34.600 Thank you, Jason. s169 00:19:34.600 --> 00:19:36.040 Really appreciate. s170 00:19:36.040 --> 00:19:40.080 If you guys haven't seen the data robot in life in real life, have you check it out? s171 00:19:40.080 --> 00:19:43.600 I will say see as very impressive performance. s172 00:19:43.680 --> 00:19:46.280 Do we have any question or want to ask? s173 00:19:46.280 --> 00:19:48.560 Okay, cool. s174 00:19:51.960 --> 00:19:54.360 Thank you for sharing the presentation. s175 00:19:54.360 --> 00:20:05.000 I wanted to ask if you are already doing kind of like providing developer kits with robots, SDKs, frameworks, etc. for your commercial partners? s176 00:20:05.000 --> 00:20:13.880 Yeah, so we haven't been working on develop developer kit, but you know, at the current moment we're building our own hardware stack. s177 00:20:13.880 --> 00:20:19.400 So, maybe at some point it's on our road map, but not as of not as of now. s178 00:20:19.400 --> 00:20:21.400 Yeah. s179 00:20:26.320 --> 00:20:36.120 I'm curious if you have ventured into education like teaching kids elementary maths or any such use case you tried. s180 00:20:36.120 --> 00:20:40.400 Yeah, so we haven't looked into education use case, right? s181 00:20:40.400 --> 00:20:50.120 But I think in the future, you know, once our full stack robotics pipelines mature, once we have our own hardware, our data collection, you know, toolkit, I think it's possible s182 00:20:50.120 --> 00:20:57.800 and I think it'll be very interesting to venture into education use cases because I also think robotics will only get bigger in the future. s183 00:20:57.800 --> 00:21:07.080 So, I think it'll be really really interesting to get children and young kids into the field and the learning how to actually train models to do tasks. s184 00:21:07.240 --> 00:21:09.160 Thank you for the presentation, really good. s185 00:21:09.160 --> 00:21:15.960 Uh my question is like is there do you guys have a plan for uh kind of taking this technology direct to consumer? s186 00:21:15.960 --> 00:21:20.560 Like I saw the use case for the laundry folding laundry. s187 00:21:20.560 --> 00:21:22.200 A lot of people don't like folding laundry. s188 00:21:22.200 --> 00:21:24.040 I think this is a great use case. s189 00:21:24.040 --> 00:21:26.960 So, is that something that you guys thinking about? s190 00:21:26.960 --> 00:21:33.400 And obviously there's a cost and all those aspects, but what's what's your plan in the long term on this technology? s191 00:21:33.400 --> 00:21:40.320 Yeah, so our long-term plan is to develop, you know, like a robot model plus hardware platform that can be deployed anywhere for anyone. s192 00:21:40.320 --> 00:21:44.160 So, that would include, you know, like going directly to consumers. s193 00:21:44.160 --> 00:21:49.080 And you know, our models today are able to just fold, you know, uh many kinds of garments. s194 00:21:49.080 --> 00:21:59.600 But uh in terms of like go to market strategy, our current focus is on enterprise use cases because I think the distribution channel is like I think it's easier to uh work with s195 00:21:59.600 --> 00:22:01.320 uh enterprise customers. s196 00:22:01.320 --> 00:22:04.440 Like we can deploy right away and iterate at customer sites. s197 00:22:04.440 --> 00:22:12.240 But for consumer product, I think uh I would imagine that it has to be very very polished and it's less tolerant for mistakes. s198 00:22:12.240 --> 00:22:28.200 And there is a lot of privacy safety concerns that we think uh we are trying to not get into at the current moment to uh unblock us from deployments because we think that the bottleneck for AI robot is getting AI and hardware s199 00:22:28.200 --> 00:22:29.800 co-working together very very well. s200 00:22:29.800 --> 00:22:38.000 And I think commercial environments provide ideal testing ground uh for companies and for the entire field at this stage. s201 00:22:38.480 --> 00:22:40.240 Thank you, Jason. s202 00:22:40.240 --> 00:22:46.200 Uh my name is Ahmed and I work in a in voice AI in in the voice space. s203 00:22:46.200 --> 00:23:00.880 And at least for consumer robots, uh where I believe humans will want to issue voice commands to robots, uh can you talk about how to integrate uh traditional voice stacks, you know, which are intent-driven s204 00:23:00.880 --> 00:23:06.640 and uh uh different kinds of models from uh uh robotic, you know, VLA models. s205 00:23:06.640 --> 00:23:07.920 Yeah, that's a great question. s206 00:23:07.920 --> 00:23:13.200 So, we think uh robot human interactions really important, and it at the end of the day, right? s207 00:23:13.200 --> 00:23:15.520 We want to develop models that are teachable. s208 00:23:15.520 --> 00:23:19.440 So, it'd be really nice if humans can speak to it, and the robot does the task. s209 00:23:19.440 --> 00:23:25.920 So, the way I think about it, so I'm not I don't have a speech or audio background, so what I can think of is like there is very mature s210 00:23:25.920 --> 00:23:27.640 speech-to-text models, right? s211 00:23:27.640 --> 00:23:43.320 So, you can use that running the on-board to translate human intent or human speech into text very quickly, and then, you know, that's where our reasoning model and our world model comes into play, because both of them are able to interpret text commands. s212 00:23:43.320 --> 00:23:48.280 So, that will allow the model to translate instruction into actions. s213 00:23:48.280 --> 00:23:59.560 But, I think in the overall stack, the bottleneck is still on the robot foundation models, because models today our models and other people's model are not ready to just execute any arbitrary commands. s214 00:23:59.560 --> 00:24:03.720 So, I think there's still a gap in terms of low-level robotic manipulation to get there. s215 00:24:03.720 --> 00:24:06.800 But, I think that's the eventual goal that we want to get to. s216 00:24:06.800 --> 00:24:10.320 Just a model that can be steerable, that can be taught by anyone. s217 00:24:10.320 --> 00:24:12.320 Yeah. s218 00:24:12.800 --> 00:24:13.400 Okay. s219 00:24:13.400 --> 00:24:14.040 Thank you. s220 00:24:14.040 --> 00:24:26.160 Um how do you think about, I guess, the split of intelligence between the general foundation models that you all are training, I guess, something like a task-specific model for something like laundry folding, s221 00:24:26.160 --> 00:24:33.960 and then kind of some of the on-site specific training that needs to happen for a specific deployment in a particular environment? s222 00:24:33.960 --> 00:24:37.360 Yeah, I think both are very very important, right? s223 00:24:37.360 --> 00:24:43.960 So, the kind of recipe that we have is, you know, we have pre-training, post-training, then we have some of the sort of active learning, right? s224 00:24:43.960 --> 00:24:48.120 I I think uh if you think about, you know, humans, right? s225 00:24:48.120 --> 00:24:57.440 You know, we have like basic level of like semantic and physical understanding that allows us to adapt any physical task very fast, and I think the best way to even build s226 00:24:57.440 --> 00:25:01.240 commercial grade robots today is by doing that. s227 00:25:01.240 --> 00:25:07.720 Because by doing a lot of generalist model training have a general understanding of physical world that's very useful. s228 00:25:07.720 --> 00:25:17.480 And what we have seen, so this is something I didn't talk about in the talk is that when the model was recovering from all kind of errors that you saw, a lot of that also just came from interpolating s229 00:25:17.480 --> 00:25:21.880 different error recovery behavior that is gathered from doing other tasks, right? s230 00:25:21.880 --> 00:25:31.560 Just because, you know, the model has seen thousands of tasks we had some data to recover from all kinds of mistakes is able to just execute that kind of like s231 00:25:31.560 --> 00:25:34.080 uh on demand on a new task it hasn't seen before. s232 00:25:34.080 --> 00:25:42.600 But if you're only specializing a model from the get-go on one task, then it's actually missing out on a lot of the physical knowledge that allows the model to be more robust s233 00:25:42.600 --> 00:25:44.160 than if you only train on one task. s234 00:25:44.160 --> 00:25:46.560 And that's what we have seen consistently. s235 00:25:46.560 --> 00:25:57.040 That's why, you know, in the beginning of the talk I also emphasize on having that general pre-training backbone and then doing post-training and active learning on top of that to get to actual commercial grade usability, s236 00:25:57.040 --> 00:26:03.320 not on just one task, but also the same recipe that can be repeated across many many tasks. s237 00:26:03.320 --> 00:26:03.840 Awesome. s238 00:26:03.840 --> 00:26:04.440 Thank you, Jason. s239 00:26:04.440 --> 00:26:05.520 Really appreciate. s240 00:26:05.520 --> 00:26:08.280 Um that's a wrap for uh data session. s241 00:26:08.280 --> 00:26:12.120 Uh if you have more questions, feel free to catch Jason after the session as well. s242 00:26:12.120 --> 00:26:14.360 Uh but yeah, this wrap our morning session. s243 00:26:14.360 --> 00:26:18.560 Uh this afternoon we have Unity, Skydio, Zoox, uh Waymo, DeepMind. s244 00:26:18.560 --> 00:26:20.480 Um so we'll catch you and see you AI as well. s245 00:26:20.480 --> 00:26:21.920 So we'll see you in the afternoon. s246 00:26:21.920 --> 00:26:24.160 Thank you, everyone. s247 00:26:39.528 --> 00:26:41.528 [music]