Building Generalist Humanoid Robots (ICRA 2026)

In this talk

1. 0:16

   Why humanoid robotics is at a turning point

   Zhu says more capable hardware and scalable learning methods are creating a new phase for humanoid robotics. He traces past cycles of progress and setbacks, including the DARPA Robotics Challenge, and points to a surge in development since around 2022. He says the field has reasons for optimism, while no final recipe for deployment exists.

2. 3:19

   Foundation models and a pyramid of training data

   Zhu describes NVIDIA's GR00T-1 as a two-system model: a vision-language system interprets images and instructions, while a diffusion transformer generates robot actions. Whole-body controllers translate high-level commands into joint movements. He frames training data as a pyramid of human video, simulation, and real robot experience, each with different strengths and limits.

3. 7:14

   Learning whole-body control and dexterity from people

   Zhu presents Sonic, a whole-body controller trained with human motion capture retargeted to specific humanoid robots. Its training used 100 million frames, more than 10,700 hours of motion data, and 9,000 GPU hours; the model can run onboard a Jetson Orin. For dexterous manipulation, EgoScale and EgoSkills use egocentric video to learn human hand movements, align human and robot data, then adapt with a small amount of robot data.

4. 12:41

   World models, deployment, and open questions

   Zhu describes DreamZero as a world action model that predicts future video while also generating actions. He says the approach improves zero-shot generalization and post-training sample efficiency, and shows a complex assembly task learned within a day without teleoperation data. In questions, he emphasizes broader deployment metrics beyond success rate, diverse pretraining data, and the unresolved challenge of combining modalities.