Jitendra Malik, professor at UC Berkeley and researcher at Meta, discusses robot learning inspired by child development at a Paul G. Allen School talk hosted by Steven Seitz.
Malik frames intelligence as a biological process rooted in the cycle of perception and action. He argues that robotics, which connects perception to action, deserves attention alongside language because movement and manipulation developed long before modern language.
Malik identifies locomotion, navigation, and manipulation as central problems. He says navigation is nearly solved, locomotion has advanced quickly, and manipulation remains the bottleneck for useful home robots, perhaps only 20% solved and five to ten years from major progress.
Robotics lacks the abundant training data available for language and vision, so Malik advocates learning from child development. He highlights learning that is multimodal, incremental, physical, social, and eventually shaped by language, while treating biology as inspiration rather than a strict blueprint.
Malik describes quadrupeds learning to walk without preset gaits, adapting to terrain through reinforcement learning in simulation before transferring to real robots. In manipulation, combining vision and touch nearly matched performance with full state information, and proprioception helped teach a robot to see step height over time.
Malik argues for hierarchical learning, building higher-level skills on reliable movement, and shows robots learning pursuit behavior and humanoid motions from video. He estimates that observation could provide about 90% of robot learning data, with the remaining 10% coming from interaction, especially for skills such as tying shoelaces or using chopsticks.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
Jitendra Malik, professor of EECS at UC Berkeley and vice president and distinguished scientist at Amazon, discusses robot learning and dexterous manipulation at a CMU Robotics Institute seminar introduced by Shubham.
Wei of Tsinghua University, Ren of Renmin University of China, Wen, Jad, Priya, Joey of Remix, and UC Berkeley professor Jitendra Malik present CoRL 2024 research on robot learning, manipulation, and perception during the Conference on Robot Learning oral session.
Jitendra Malik, Berkeley professor and part-time research science director at Meta, talks with Peter on The Robot Brains Podcast about sensorimotor learning, robotics, computer vision, and how AI research develops.
