Robust Manipulation in the Real World: Principles and Applications - Dr. Yifan Hou 11.21

In this talk

1. 1:10

   Why manipulation fails

   Hou contrasts recent progress in robot locomotion with the persistent gap in manipulation. Contact-rich tasks can fail after small position or force errors, such as missing a light bulb socket or failing to get a blade under an object. He argues that reliability requires handling uncertainty across sensing, models, actuation, and external forces.

2. 7:35

   Use mechanics to prevent failures

   Hou groups immediate manipulation failures into excessive internal force and unintended changes in contact. His hyper-servoing approach adjusts compliance to reduce kinematic conflicts, while a wrench-space method selects a desired contact mode using force control. He says these approaches can support delicate actions even when the robot’s model of an object’s size is inaccurate.

3. 16:53

   Learn force-aware behavior from demonstrations

   Hou describes imitation policies that use force feedback and produce compliance settings as well as position commands. A simple profile makes the robot compliant along the current force direction and stiff in other directions. A handheld gripper with two compact CoinFT sensors also lets people collect force-informed demonstrations without using a robot.

4. 23:41

   Improve policies with gentle corrections

   Hou presents Compliance DAgger, which lets a pretrained policy keep running while a person nudges the robot through a compliant handle. This avoids abrupt control switches and produces correction data for a residual policy. On four tasks, 50 to 100 correction episodes brought performance close to 100%, according to Hou.

5. 30:08

   Extend demonstrations and bootstrap new tasks

   For dexterous hands, Hou’s DexUMI exoskeleton matches the robot hand’s fingertip motion so demonstrations can transfer within the motion range shared by person and robot. He also proposes generating varied contact examples in simulation, then using those data to build a policy’s basic capabilities before refining it in real-world deployment.