CMU PhD student Tyran and UC Berkeley PhD student Haru discuss NVIDIA internship projects VIRAL and Doorman, which use simulation to transfer humanoid manipulation policies to real robots, with RoboPapers hosts Chris and Jeff.
VIRAL trains a privileged reinforcement learning teacher, then distills it into a vision-based student. A Unitree T1 uses head-camera RGB and proprioception to move objects between tables in a nearly half-hour demonstration, with no real-world training data. The team used Isaac Lab rendering and visual randomization across as many as 64 GPUs.
Tyran says reward tuning consumed more than 80% of the team’s effort. A reward that only encouraged placing an object in a tray led the robot to throw it from several meters away, so the team shaped rewards toward stable behavior that could transfer to hardware. Reference state initialization helped the teacher explore, while delta actions and a pretrained whole-body locomotion policy supported manipulation.
Doorman applies the approach to opening doors, a harder task that is less straightforward to teleoperate. After teacher training and student distillation, the vision policy reached roughly 50–60% success; a third reinforcement learning stage raised results to the high 80s or 90s. The team randomized door geometry, handle and hinge physics, and visuals, and used RGB observations without door markers or localization estimates.
The researchers say simulation can provide broad generalization across object positions, table heights, lighting, and other environment variations, but building articulated assets and tuning rewards takes substantial effort. They see vision transfer as less troublesome than reward design and contact physics. They describe simulation as valuable for constrained tasks and policy evaluation, but not a complete solution for robotics.
The discussion turns to how robotics should measure difficult tasks and explain their value to the public. The researchers see humanoids becoming more useful as hardware, control, and infrastructure improve, especially if robots can learn from large amounts of human data. They also discuss combining physics-based simulation with video models, while noting unresolved questions around contact, sensing, and controllability.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
Yan Han, an author of FALCON and an applied scientist intern with Amazon Frontier AI, discusses force-aware humanoid control with Flor and the RoboPapers audience.
A Booster Robotics presenter explains how to assemble, operate, charge, and repack the T1 humanoid robot.
