Robotics Has Been Stuck for 70 Years — Deepak Pathak, Skild AI

AI Engineer · 28 min · 336 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:12I think uh all of you know here that how much progress has AI made in the last five years.
00:17Uh seems like it's more than uh last 100 years of progress.
00:22And what is happening especially in AI is AI came for language then speech audio video and everybody's excitement is next is robotics okay and this excitement has is going on peak uh this year uh
00:36for some reason which is still beyond my uh understanding and you can also see uh leaders like Jensen talking about physical AI as the next frontier of robotics.
00:46So it seems like if you open Twitter or LinkedIn, it seems like robots robotics is already here and we'll have home robots in our homes sometime by December uh as uh as it's being said by multiple people.
00:58But I'm here to highlight that robotics has been almost here for the last 70 years
01:06[sighs]
01:06now.
01:08Those of you who got into robotics in the last four or five years, go and take any any any course in robotics and if you can distinguish the videos in robotics from 30 years ago versus today,
01:20uh I owe you a dinner.
01:23So to give you an example, let's look at this robot.
01:26So this robot here is an image and the goal of this robot is to look at this image of blocks and arrange these blocks in front of it so that it looks the same pattern as in the image.
01:37Okay, this is an extremely simple task.
01:40But if you think about this, it's a 3D block.
01:41You have to look at understand 3D from each side.
01:44And the robot can do this pretty precisely.
01:47Any guesses how old this result is?
01:51Anyone?
01:51It's a guess.
01:52You can make any guess.
01:53Huh?
01:5440 years.
01:55Okay, that's the highest.
01:57[panting]
01:58This is from 1960s.
02:01This is way before there were computers.
02:03Uh, okay.
02:04So, this is this is called MIT copy demo.
02:06Uh, this was the start of uh this is this predates AI.
02:10Uh, as you know, today look at this one.
02:17One exhibit at the Nuclear Congress in Philadelphia has the perfect formula for
02:22what's happening here.
02:23The guy behind the scene is controlling these robots, a leader follower system.
02:27And if you look at any big lab, any big company, any big academic lab, how they get data, they use teley operation system, which follows the exact same principles.
02:38Any guesses for the year for this one?
02:42Now, everybody would correct aggressively on the year, but this is from 1957, 68 years ago.
02:49Okay?
02:51Ask anybody in your family who is older than actually they may not be around like so because this is for 60 you have to ask somebody who is 80 year old what was the state of technology at that time like RAM
03:03for few KB of RAM you would have a size of a room uh like this big of a setup and this is from that time when people could make it work
03:11on not just this you keep coming back every decade the videos look as cool like here is robot a humanoid juggling playing foosball uh with human and this is also from you can see this is all they were all predate deep learning this from 8 years ago so this robot is a result from Berkeley can clean up
03:29the whole table can arrange items in a in a in a box this was done by one grad student with a single GPU machine nothing more than that okay
03:38now this picture of and if I colorize all these videos from past and apply genai filter for the modern voice you cannot tell that this
03:47[laughter]
03:47video is from 19657 or this is from today And these are not even the oldest results the oldest ones go to 1940s.
03:54Okay.
03:55So what is this that if you look at last 70 years tech every other technology every other technology has come way far language understanding computer vision mobile phone chips
04:08why is robotics is stuck in this primitive age uh for for this long and the reason is uh a general brain.
04:14Robotics has always been approached as a hardware problem from ground up and this the reason that everything around robotics has progressed but robotics is still stuck in the same land from the last 70 years.
04:25So there's a very famous paradox called Morave paradox.
04:28I don't know if you know about Moravec.
04:29He was also a CMU professor and I'm also CMU professor so have to quote him for sure.
04:34He was one of the founding figures in in AI.
04:36After 30 years of working in robotics he arrived at a very simple conclusion.
04:41Hard is easy, easy is hard.
04:43Okay.
04:44Whatever human believe to be hard is extremely easy for computers and vice versa.
04:48And you can see it happening right in front of your eyes.
04:51You would say doing math is hard.
04:53Math Olympiad gold medal is really really hard.
04:56Climbing a stair oh super easy.
04:59Now look around you in technology where we are in terms of what has been solved and what's not being solved.
05:04So this is robotics for you.
05:05Okay.
05:05It's very it is not yet another application of AI.
05:09It is what AI was founded for in the very beginning and has made very little progress towards.
05:14This is not yet another application where deep networks can come in and attack the field.
05:18This is has to be thought of with fundamentally first principles from the ground up.
05:23This is a very famous example of where you can have a you know a computer beat uh Gary Casper on chess in '90s but you can you still don't have a a a computer or robot that can pick up the chess pieces and arrange them on any chess board.
05:38Now, so how do we go about solving this?
05:41Okay, so on a more positive note, uh we have a magic sauce for AI success, right?
05:49You get big data set, you train big models and magic happens.
05:53Okay, now we know this template is working well very well in many topics.
05:59Okay, now can we apply the same recipe to robotics?
06:02Well, it's not uh directly applicable because we have no data.
06:07There is no internet of robotics data and as I said earlier you can go and collect data uh manually on the robot called telly operation and as you notice teley operation is not 5 year old thing this is a 68 or 70 year old thing so
06:22the the funny part is if you go back and you look at the whole evolution of GPD3 GPD4 models go back to GPD3 uh three years ago seems like a like a lifetime ago
06:33GPD3 only began working and caught people's attention when you could train those models on trillions of tokens.
06:40So GPD3 was already trained 30 trillion plus token and today hundreds of trillions of tokens and if you collect data by manually by tell operation it takes you about 1 minute to get one example.
06:49You can do the math if you hire all of US population it will take you more than a century to reach the same scale as of GPD3 uh which is like uh and nobody uses GP3 today.
07:00Okay.
07:01So this is very slow and expensive.
07:02So in robotics I would argue nobody has really scaled robotics yet and we are very far from talking about scale in robotics with the way other uh other areas have seen scale.
07:14Okay so this is where uh we have been focusing on uh our scaled is about uh 3 years old but uh I have been working on the problem for more than a decade
07:22uh that's all I've done in my career uh nothing else.
07:25Uh so at scale what our thesis is is to build what we call an omniodied intelligence any robot any task one brain okay any robot it can be a humanoid it can be a quadriped it can be a robotic arm on a conveyor belt or a dextrous hand doesn't really matter
07:42this even this hypothesis is way more general than uh one would argue humans are because we control our own body and why do we have to go so general well the argument is in robotics
07:54there there is no data anyway.
07:56So I can't pick and choose which hardware do I use data from.
07:59And we should be able to use data from any kind of hardware, any kind of task, any kind of scenario.
08:04And that's the only way to truly achieve the scale of uh of what language models achieved uh three years ago.
08:10Okay.
08:10So the goal here is this uh this whole idea of any robot any task one brain.
08:15And through this talk I'll hopefully convince you why this is the this is the way to go towards robotics.
08:20Okay.
08:20So this is the uh rough intro but before I go into the more details let me show you just a teaser result.
08:28So in this result, every single robot uh it's from a different company, different hardware and they're all controlled by uh skilled brain whether it's humanoids going up and down stairs, any kind of scenarios.
08:42And these are not new results.
08:43They're like a couple of year old uh results in here.
08:47Robust to dust disturbances.
08:51You can put them zero shot in new scenarios.
09:02So the idea here any every single robot in this video takes any action anywhere in the world the brain behind the scene
09:12[music]
09:12improves because it's an omni-body brain.
09:15Yeah, this is really hard right because like eggs are okay.
09:32So that's a teaser of what we what we work on.
09:34So I want to make this talk more scientific and more uh more informative than a company ad.
09:39So I'll talk about uh how do we scale data in robotics.
09:42Okay, let's take a tour back as to how have people addressed this.
09:46uh I'll take a look back at my own career and my hypothesis for data has been changing over the years.
09:52Okay.
09:52When I began uh uh working scaling things the idea was you can have robots you can collect data for robots manually but it is too slow and robots should be allowed to collect data by themselves.
10:03So we had this whole idea of curiositydriven exploration.
10:06Uh allow robots to explore in a curious way in the environment and it has a lot of good parts like no human required.
10:13Robots can go and keep playing collect more data.
10:15We called it play data at the time.
10:17It's very rich because it has force sensors and joint angles.
10:21But the difficulty is it is only in the physical world.
10:24So it's very difficult to scale.
10:26E Google was at it at the time.
10:28Uh h having hundreds of robots.
10:31Even for Google it's too expensive and too slow to scale.
10:34The other idea is again telly operation.
10:36As I said it's an old idea but again the same issues impossible to scale because now you don't need robot you also need humans.
10:43Uh so it's even more expensive and diversity is very limited because even if you put a robot in one setup with a human you can get thousand examples but they'll all be in the same setup.
10:53So what you ideally want carry the robot to new home every day in new scenario and that's just impossibly hard to scale.
11:01Then we had major breakthrough in learning from simulation.
11:04This was one of the first result where uh one could show deep reinforcement learning uh trained in simulation transfer to real robot.
11:13This is one of one of the award-winning paper uh at the time and now it's used in every humanoid uh every company out there.
11:20uh this was from our lab at Berkeley and CMU.
11:23But again they there are pros and cons.
11:25It's very easy to scale but diversity is hard to get because you have to engineer every scene in simulation manually.
11:31So if you're listening to this like there is no single answer I'm coming to.
11:35Uh there is no uh no no uh no single solution.
11:39This is uh another work we did earlier.
11:41This is all before skilled learning from human videos.
11:44You watch the human do things and then robot copies this.
11:47Again high diversity.
11:48You can use videos from YouTube etc. Highly scalable but it's a very poor form of data because it's very far from robot.
11:56Okay.
11:57So what is the solution here?
11:58The answer is there is no there is no golden path.
12:01You have to think about data in the context of different features.
12:06And in my opinion there are only three features that matter in in robotics.
12:12scalability, diversity and how close you are to your robot uh uh robot joint angles.
12:18Okay, scalability means can I quickly scale it across scenarios.
12:22Diversity means can I get diverse data because just having 100 trillion tokens is completely useless if they're all in the same environment and same scenarios.
12:31Okay.
12:32And closeness to robot mean how far are you from the ground truth of robot own joint angles.
12:38So if you look at simulation very scalable low diverse uh uh but moderately close to robot human videos are very scalable and diverse but very far from robot data you have to learn to map the human to robot
12:49this other two teleop and the and the manipulation interfaces they are kind of in between and you can see here uh right now around the world if you look at different companies they're all focusing on one of these approaches
13:01most of them are on teleop or or yumi setups very few on everything else and but there There is no golden answer like they every each one of them have a downside
13:11but the golden light here is that they are all complemented to each other like they are not their cons are not exactly matching from each other.
13:20So this is where uh the recipe that we have converged to over the years is to realize separate the training into two parts pre-training and post- training but that's not surprise right but how do we pre-train you want to pre-train using data which is highly scalable and diverse but maybe low quality so simulation human videos etc. So
13:37that's how you pre-train and then for post training you use data from telly operation.
13:42Now what is teleoperation data?
13:44It's very high quality but low in amount.
13:46Okay, high quality low in amount.
13:48It exactly reminds us of the recipe in language models.
13:51You pre-train data on the internet then you post train for coding for uh uh for your own company etc. But in robotics there is one more bucket which is deployment data
14:02and deployment data is highly scalable once deployment scaled.
14:07So over time this data will take over everything else.
14:11And what we are trying to build is what we call this data flywheel which goes which takes this data from post training time and puts back in pre-training.
14:20And now you can see why this idea of omniodied brain is extremely important because this is very unlikely that you have only one robot only one version deployed forever
14:30in every task around the world.
14:32This never happens in any area of hardware except chips because they are very hard to manufacture.
14:38Uh and you can still see even in the chips uh inference chips are coming left and right for many companies these days.
14:43So in hardware this is never the case.
14:45you have only one version of the robot, one shape, which is why omniodied intelligence is the enabler of what we call a deployment data flywheel.
14:54Okay, so let's now look at a few results uh very quick.
14:59Now this brain is extremely general.
15:00So we can do variety of tasks very quickly.
15:03You may have you have se you may have seen many tasks like laundry folding etc. It's very popular task in Silicon Valley for some reason.
15:10uh and uh and the and the argument here is uh when have you ever thought while folding a t-shirt that if I miss my hand by 1 cm my t-shirt fold will be a blunder
15:22you don't think like this people don't even think while folding they just hold anywhere you just do something so that the tolerance for error is extremely extremely high then why is this task hard
15:34anybody why why do people get why do people get fooled into believing this task is Let me put it that way.
15:42Fabric, right?
15:43Why is fabric hard?
15:46Simulation.
15:46But nobody's using simulation anyway.
15:48This is all from telly operation.
15:49Why is this hard?
15:51You know why is it hard?
15:52Because it is hard for classical version of robotics.
15:56Classically in robotics, people would model the whole physics, create models by hand and then do this.
16:01It's very hard for that.
16:02But for deep learning based robotics, this is the easiest task possible because it has high tolerance for error.
16:08So it is completely uh now so many companies focusing on this task.
16:12Now what is hard is I would say something about like let's say this task.
16:17If I ask you before seeing this video is this task doable without having hands very likely half the people will say no because it requires putting like how many of you have lost airports?
16:28Uh like and in our company there's a channel called right airpod because people keep losing their right.
16:34Now in this case the robot does not have hand it has gripper.
16:37So here the task is very hard for the gripper.
16:41So it requires a higher level of intelligence because the arm has to go and orient itself to pick up the airpod in the right manner such that it can be inserted because the grippers can only close up and like like this.
16:54They're parallejo grippers.
16:55So you cannot turn the uh the the the airpod at the very end.
17:00Now what you are seeing here these are not real airpods.
17:02They are fake ones from teu uh like $5 $10 each.
17:06So they don't have a magnet inside.
17:07So the robot has to really work hard to put this inside properly because there is no magnet to pull it uh easily.
17:13So these are all fake uh fake ones.
17:15This is even harder than it appears uh in the video.
17:18You can also uh like once you build the general brain behind the scene, you can also go and learn it from variety of just human videos without actually having any finetuning data at tell operation time.
17:29So in this scenario what we did, we trained on human videos like this like egocentric videos.
17:34we are now trying to transfer it to more third person uh uh videos and if third person works you can learn from YouTube any any kind of open source data.
17:43So in this case we see the video and then we add less than 1 hour of robot data.
17:47So very quick uh transformation and then the it can transfer to humanoids.
17:52Now these ones have hands.
17:54Hands or no hands is a big debate.
17:57People often use hands as an excuse as to why robotics is not here but that's not the case.
18:02It's always the intelligence uh behind the scene.
18:07We don't deploy hands right now because there are none available which can be deployed in factories.
18:12They all break within 100 yards.
18:13Uh pick anyone.
18:17If you have better hand uh I would love to buy 10 units ASAP to test.
18:22So it's robust to scenarios.
18:25Now the idea is you can do many tasks by watching humans.
18:28And the reason we can work with very less data which is less than one hour of robot data is because robot imagines things in its own head and tries to multiply the learning from for many scenarios.
18:39So what you see here is a completely fake video uh you can call it robot's dream.
18:43So it's inside the robot's own model where it can imagine scenarios uh in so this is not real data.
18:49This is all fake uh from own robot's own model.
18:52You can transfer it to you know more complex task and even lower cost hardware.
18:57So this is a egg uh uh egg making like omelette making task.
19:02Uh now here the these this whole setup costs about $4,000.
19:06So extremely cheap arms uh compared to the uh what you see out there.
19:11And if you notice here this setup is too cheap to even have any sensors.
19:16So the only sensor here is just a camera.
19:19No force nothing else.
19:20And the robot can do task which require forces uh from vision.
19:25Now I'm not saying that's a future like you should not be doing this.
19:27I'm sure the sensors will improve but
19:31from the existing sensors we are way behind than where we can be from just intelligence perspective.
19:37Okay.
19:38So this can keep on going.
19:40This is my adviser from Berkeley.
19:42He did not believe the robots can do it.
19:43He just kept standing for like uh 1 hour and the robot kept making omelette.
19:47Uh and and the interesting part here is this is fully end to end system.
19:51There is no state machine nothing.
19:53So the reason robot is looking egg because there is an empty plate in the front.
19:56I don't know why it's fluctuating.
19:57It's not in the there's no cut in the video.
20:00I I assure you of that.
20:01It's HDMI problem.
20:02So as soon as you remove the plate uh sorry I don't know why this is happening.
20:08Yeah.
20:09So there's no state machine when the robot starts when the robot ends.
20:11It's all automated and it's very robust to even disturbances.
20:15You can change objects around.
20:16Uh you can add these are all unseen and unseen objects.
20:20Uh only the pan and the gas stove is same.
20:22Everything else is different and the robot can keep on going.
20:25So you can get basic robustness.
20:27So this was this is all trained with less than 10 hours of data.
20:30Uh so it's very low data to be learning this robustness and it's coming from the base model behind the scene.
20:37I'm skipping uh this in the interest of time.
20:39So unlike traditional robotics pipelines where you have planning, mapping etc. This is an end toend brain.
20:46Now end to end is heavily used term in in many areas of AI.
20:50But when I mean end to end, I really mean end to end.
20:53It reads directly from the cameras and it applies power directly to motors.
20:58So we use nothing uh in between except just a P at the very end to go from 100 to 500 Hz.
21:04But P is not robotics contribution.
21:06This is before it predates to like World War II or something.
21:09Okay.
21:10Now the the the interesting part is vision for us is yet another input.
21:15So nothing that crazy about it.
21:17So if you have seen, you may have seen a lot of lot of results of robots dancing, doing karate, kung fu, backflip.
21:24Go back and think how many results have you seen of robots going up and down stairs.
21:30Very few.
21:31Like even the top companies out there in humanoid, they have not shown beyond one sample stair just to check the tick tick box.
21:39You do.
21:39And people talk about humanoids are coming, China versus US, all of this is sort of BS.
21:45If humanoids cannot climb stairs, what's the point of having legs?
21:49All right, that's the only reason why you have legs.
21:51Now, this is actually a Moex paradox add action.
21:55Left one actually is much easier for a robot.
21:58A backflip, dancing, etc. is much easier for the robot.
22:01And you may think, okay, why is this paradox exist?
22:04Think one level deeper.
22:05When a robot is doing a back flip, it has to only know about its own body, nothing else.
22:13Right?
22:13So it's and when everything is known or fully observed that's what computers are good at because they can do they can simulate every possible uh uh setup but the right one even simply climbing on stairs requires seeing the stair at the first at the first place or what's the height what's the width
22:32you don't measure height and width exactly but you adjust to all the disturbances and that requires vision so right one requires understanding and that's why it is hard and you don't see any of this but for us it's just get another result it doesn't really matter so we And uh we put this result out one and a
22:46half year ago.
22:46We shot this like two and a half years ago.
22:48Uh but uh like this robot can go any scenario.
22:52It is not stumbling on things.
22:53It intentionally steps over things and there is no mapping or planning.
22:57It doesn't make any 3D map of the surroundings.
22:59It's all operating from camera on the torso.
23:02And you can put this in any kind of setup, any kind of stairs.
23:05Uh I'm going faster here.
23:07These are, you know, uh fire escape stairs.
23:11It's a bit weird.
23:12Fire escape stairs are to be used when there is fire in urgency and they are the hardest in every building.
23:17Uh so we are testing in all in all those setups but you can put this in anywhere you can disturb it and stay doesn't really matter.
23:23This is superhuman capability because robot cannot see it's being pulled.
23:27So it's a surprise factor for the robot that you're pulling the violet while climbing.
23:31And again it's the same model everywhere in all these scenarios.
23:35So very easy although easy parkour does look fun uh uh does look fun and people often find like uh this is uh you know the robots can do all this but this is
23:48so much easier than what I showed earlier and even if you see these videos right now many of you will find this more impressive even though I'm telling you this is easy it takes half an hour to train this while the previous one it takes much longer uh and much more difficult to train
24:03so you can take this Based model and you can transfer to variety of tasks very quickly.
24:08So here this is a very old result from one and a half year ago uh for inserting LAN cables etc. We have very advanced systems now but we are deploying these models already across variety of application.
24:18So for instance because to to build the data flywhe the hardest part is the deployment itself.
24:24There are so many other issues beside intelligence and hardware that come up when you deploy robotics.
24:29It is not the same.
24:30This is not same same as deploying chat GP over an app because you have to work around many of the and I think Skyio gave a talk before this and they can talk to you about all day about the hardness of deployment.
24:42So we have to start deployment now so that we can have this data flywheel in a foreseeable future and I'll give you a few examples of deployment we are doing.
24:49So one of the examples with Nvidia like Nvidia is opening their first factory in Houston.
24:54uh and their GPUs what you use right now they're all built outside US in in Taiwan mainly and it's all done manually over there and uh humos are really efficient and really uh really good at making these things
25:05but sustaining cost and scalability and throughput here in US it's very hard to maintain without this labor force so here we are automating uh this GPU assembly for them this was we did a live demo in Nvidia
25:18GTC this is deployed in factory already uh last week so it's already live but the interesting part I want to highlight this is a very traditional factory task but if you look at the the whole setup
25:32this is extremely randomized extremely noisy and you go to any factory traditional setup they're extremely clean so here the robot the same brain can not only do very precise task it can be very robust to any kind of disturbance which means now these robots
25:48do not need to be in a cage uh and they can be uh just deployed as is on on these factory lines without any change to any factory line alongside humans and wherever there is humans there is mess.
26:00Okay, you can put the same brain to delivery applications.
26:02So here the robot delivers from the from the truck to the to the door.
26:06It has to find where the front door is.
26:08So it's a common sense problem.
26:09It's not mobility problem.
26:10And we already have been delivering uh packages to with working behind the scene with many partners to delivering packages to front door of the houses uh in these areas.
26:19same setup works across warehouses and all this.
26:22Um now just to close the talk uh close the topic I mentioned this is an omnibody brain and I hope it's very clear why it's extremely essential to have an omnibody brain to build data flywheel
26:34but there's one extra benefit and the benefit is your robots may break over time and uh or the safety uh scenarios and safety is a byproduct of this omnibody brain because if you have a we can put the I'm just skipping very fast here so you can see the online this is all online
26:49uh but we can put the same brain across each of these systems each of these robots and in this case we did not even train on these robots.
26:58This is completely zero shot transfer to all these humanoids and all these things and even if the robot breaks like here the leg breaks it can recover within milliseconds because when your leg breaks
27:09your three-legged robot is a new robot.
27:12So it doesn't really matter what's the shape of the robot is anything changing here we disabled the legs of the robot it learns to learn it learns to walk on two legs in just three trials.
27:21So what you are seeing on screen is all the training for the robot that they are that that is happening in these systems.
27:27So it's it adapts in few milliseconds to uh uh 30 seconds or so.
27:32And like one example here it's it's going on wheels.
27:34You jam the wheel it starts walking.
27:37So this is a another byproduct of omnibody brain.
27:41Safety takes a very different meaning uh with these kind of models.
27:45when the robot can fly like or sorry when the robots can walk or or operate with only half the body.
27:50I removed some of the gory scenes from this but we also cut the robot in half and it can still work uh with both the two halves separately but for that you can go to YouTube uh and that's all I have.
28:00Thank you.