Roger Chu, a PhD researcher at UC San Diego working with Professor Shaolong Wang, joins RoboPapers hosts Michael and Chris to discuss using task-specific human demonstrations to improve humanoid robot learning.
Chu asks whether human movement recordings can add to robot training data, which is costly to gather through teleoperation. Their approach records people performing tasks such as moving a bottle into a basket, with hand and finger positions tracked in 3D. The recordings focus on defined tasks, with clear episode boundaries and annotations, rather than unstructured daily activity.
The team gathered about 30,000 demonstrations, nearly four million frames at 30 frames per second, across varied people, objects, table heights, placements, and settings. Chu estimates that 20 to 25 hours of recordings took roughly 30 to 35 hours of human effort, and says volunteers needed no special operating skills. They built a Vision Pro app to capture video and ARKit head and hand tracking; Quest tracking could jitter when fingers were occluded.
The policy uses a common representation of wrist poses, hand keypoints, and head movement for people and robots, without requiring the model itself to perform inverse kinematics. Human demonstrations are slowed by a factor of four because people complete tasks much faster than the robot. On the Unitree H1, combining human and robot data improved performance in unfamiliar object placements and backgrounds, and helped transfer learning to a related humanoid using just 20 robot demonstrations.
Chu says human data can help generalization, but scaling it alone is unlikely to solve every learning problem. He sees task-focused, egocentric recordings with accurate 3D hand information as useful for pretraining, while broad internet video is harder to apply because task retrieval and reliable 3D pose estimates remain challenging. The team is exploring language-conditioned policies, mobile and longer-horizon tasks, and more dexterous manipulation.
Chris highlights the collection rate and the value of varied scenes, while the hosts discuss the cost and tracking quality of headsets. They note that collecting video is not the same as collecting usable training data: accurate tracking and task alignment matter. Chu’s core point is that carefully captured human examples can complement robot data, especially where robot experience is sparse.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
Modi, a PhD researcher at Shanghai Innovation Institute, joins RoboPapers host Mike to explain EgoHumanoid, a method for combining human demonstrations with robot data to train humanoid loco-manipulation policies.
Ria, a USC assistant professor, and her student coauthors discuss their Humanoid Everyday dataset and humanoid manipulation research with the RoboPapers hosts.
NVIDIA Omniverse demonstrates how to use Mobility Gen in Isaac Sim to record robot trajectory data and render it from the robot’s camera perspective.
Lee Yi, an assistant professor at Tsinghua University and coauthor of R2S2, discusses whole-body humanoid reaching and teleoperation with the RoboPapers hosts.
PRO ROBOTS’ main presenter covers humanoid, service, industrial, and healthcare robots at China’s 2025 World Robot Conference, with demonstrations and interviews from exhibitors.
Sergey, host at Supercar Blondie, explores Unitree Robotics’ quadruped and humanoid robots at the company’s factory in China with Unitree representative Yolanda.
