Data-Centric Robotics: What Data Do Robots Really Need? | RSS 2026 Workshop

In this talk

1. 3:46

   π0.7 combines diverse data with detailed labels

   Balakrishna says π0.7 brings many dexterous tasks into one model, including laundry folding, espresso making, and food preparation. Its training uses varied demonstration and evaluation data, with language descriptions, quality ratings, and subgoals to explain how actions were performed. These labels can help the model learn from imperfect examples; in laundry-folding experiments, adding more mixed-quality data improved results when the model had metadata.

2. 38:26

   Dexterous skills need multiple data sources

   The next presentation describes combining simulation, teleoperation, and human demonstrations to train dexterous robot hands. Simulated policies provide reusable finger skills, while human operators control higher-level movements and gather real-world tactile data. Another approach uses simulation to turn human motion into physically plausible robot actions, and a thin tactile glove captures contact signals during human demonstrations.

3. 1:10:54

   Human experience can transfer through diverse robot training

   A presentation on embodied human data describes using hand tracking and first-person video to train alongside robot demonstrations. The EgoVerse dataset combines task-focused demonstrations with 1,200 hours of varied activity, including cooking, cleaning, packing, and laundry, with 3D tracking and language annotations. Experiments across multiple institutions found that combining aligned human data with free-form examples improved robot performance on unfamiliar tasks.

4. 1:45:04

   World models help robots test possible actions

   Wenlong proposes using 3D world models to predict how scenes change under robot actions, enabling evaluation of possible behaviors before execution. PointWorld uses RGB-D observations and robot geometry to predict scene motion, with about 500 hours of real and simulated interaction data. He argues that such models could support ongoing robot learning by helping systems compare actions and adopt behaviors that work.

5. 2:24:06

   Panelists discuss quality, coverage, and evaluation

   The panelists describe robot teleoperation as valuable for learning precise skills, while human, simulated, and other data sources can broaden behavioral coverage. They emphasize that data sources work best in combination: aligned examples can help connect human behavior to robot capabilities. The discussion also highlights tactile and force sensing for dexterous tasks, and quality checks for issues such as dropped frames and poor execution.