Tony Zhao on Training Home Robots With Human Data

In this talk

1. 7:22

   Scaling robotics with wearable data

   Zhao says his Stanford research suggested that more data and compute could improve robotic dexterity, including tasks like tying shoelaces. He points to his co-founder’s UMI work as a way to collect robot-training data with inexpensive gloves, without relying on costly robot teleoperation.

2. 11:53

   Matching the glove to the robot

   Sunday’s glove and robot share the same manipulation surfaces, materials, and stiffness, so actions recorded with the glove can transfer directly. Zhao says the high-fidelity data supports success rates he cites at 99% to 99.5%, while reducing the risk that dataset biases limit performance.

3. 16:19

   Building an affordable, useful product

   Zhao argues that simpler hardware can work when capable software compensates for its limits, and that manufacturing and supply-chain work are central to controlling cost. He sees a robot’s price as tied to its capabilities: it may begin with a few tasks and expand over time, much like a phone gains value through software.

4. 23:59

   Home tasks can prioritize quality over speed

   Zhao says Sunday is not optimizing for speed first: a robot could tidy a living room over several hours while its owner is away. For home use, he prioritizes adding reliable capabilities and finishing well; industrial robots face tighter throughput and return-on-investment demands.

5. 30:11

   Generalization is the deployment challenge

   Zhao distinguishes impressive demonstrations from systems that work across unfamiliar places and objects, calling broad deployment a major challenge for robotics. He argues that homes and factories both require adaptable robots, and that learning in varied homes could prepare robots for work in industrial settings.