Ep#85: Tutor Intelligence

In this talk

1. 5:49

   Building a real-world data factory

   Tutor says hundreds of Sunny robots, a two-armed platform related to Cassie, collect data in its Watertown headquarters. Its DF1 setup is intended to mirror a production fleet, while the TI0 vision-language-action model is trained on DF1 data. The team’s approach is to improve policies through ongoing robot operation, data collection, and human feedback.

2. 11:58

   Teleoperation designed to scale

   Video-based VR teleoperation made operators wait for visual confirmation, slowing tasks and producing less confident movements. Tutor developed Pleyop, which maps an operator’s hand motion in 3D to the robot’s gripper and can provide haptic feedback. The team reports better task completion time and success rate with 3D teleoperation, and uses task reversals and policy interventions to reduce operating demands.

3. 19:31

   Making mixed-quality data useful

   Different operator speeds can give a model conflicting actions for the same observation. Tutor uses velocity normalization to speed up moderate and fast motions while preserving slow and contact-rich segments. Its labeling system gathers rubric scores, pairwise comparisons, and thumbs-up or thumbs-down feedback, then uses the highest-rated demonstrations for further training.

4. 28:09

   Reward models and the deployment loop

   Tutor combines human labels into rewards that account for task quality and time, and trains reward models to distinguish better demonstrations from failures such as missed picks. The team says its labeled reward signal tracked errors more clearly than the off-the-shelf RoboMeter model in its comparisons, though its reward models have so far been task-specific. DF1 is a test bed for systems that can mix autonomous operation and teleoperator intervention before deployment to customers.

5. 43:19

   Choosing tasks and hardware for economics

   Tutor’s strategy is to integrate hardware, manufacturing, teleoperation, data, and software so its robots can improve and eventually do useful work in production. Sunny is not yet generally available, and the team is exploring warehouse tasks; it says the DF1 data and models may not transfer directly to production. Tutor estimates that even 50% autonomy could improve the economics of robot deployments compared with relying on teleoperation alone.