2025 Winter Robotics Colloquium: Peter Stone (Professor of Computer Science, UT Austin)

In this talk

1. 0:49

   Research background and human feedback

   Stone surveys work ranging from RoboCup and service robots to autonomous driving, then focuses on ways people can help robots learn. He traces this direction to Brad Knox’s TAMER system, where people rated an agent’s actions, and describes later work that learns from demonstrations, interventions, preferences, and facial reactions.

2. 15:36

   Adapting navigation planners

   Stone explains why standard navigation systems can struggle in narrow passages: settings that help in tight spaces can make a robot unnecessarily slow in open areas. APPL keeps the established navigation stack and learns its local planner settings for different environments. The approaches use joystick demonstrations, corrective interventions, simple positive or negative ratings, or reinforcement learning in simulation across 300 BARN environments.

3. 41:37

   Learning manipulation from one video

   Stone presents ORION, which learns a manipulation task from one RGB-D video, a stationary camera, and a short list of relevant objects. It tracks hands and objects, identifies changes in contact, and transfers the demonstrated object movements to new layouts and object instances. Across seven tasks, success was generally around 60%, with errors including tracking and grasping failures.

4. 56:33

   Practical lessons and open questions

   Stone argues for retaining well-engineered robotics components and learning the parts that remain difficult, such as planner settings or task rewards. He notes that APPL assumed people gave accurate feedback, while separate preference-learning work models noisy input. In the discussion, he says ORION could use demonstrations by robots if their hands can be tracked, and identifies obstacle-aware arm planning as an unresolved extension.