FALL 2021 GRASP on Robotics- Dorsa Sadigh, Stanford University

In this talk

1. 8:31

   Learning compact models of interaction

   Sadigh argues that agents do not always need detailed, nested models of one another. They can instead learn a compact representation of a partner’s strategy, which she calls a convention, and use it to coordinate. In a two-arm air hockey example, LILI learned whether an opponent aimed left, center, or right from past trajectories.

2. 19:33

   Using influence to improve coordination

   Sadigh extends LILI from predicting another agent to shaping its behavior over repeated interactions. In the air hockey experiment, the robot favored blocking toward the left; its success stayed near 90%, while the opponent sent the puck left more often. SILI applies a related idea by rewarding the robot for keeping its partner’s learned strategy predictable, which can reduce the difficulty of adapting to a changing partner.

3. 32:27

   Learning from human preferences

   Sadigh turns to ways of learning from people when robot demonstrations are scarce. One approach asks people to compare two robot trajectories, then selects follow-up questions to narrow down the robot’s possible reward functions. In a simple driving simulator, 70 binary answers helped a car learn to stay on course and avoid collisions; she also describes extending preference learning to nonlinear and differing user preferences.

4. 51:59

   Making use of imperfect data

   Sadigh describes learning from ranked trajectories and demonstrations that vary in quality or come from different robot bodies. Confidence-Based Imitation Learning, also called TAIL in the results she shows, learns confidence weights for demonstrations and updates them using a small set of trajectory rankings. The method aims to rely more on useful examples and less on poor ones.