Ep#89: Contact Grounded Policy

In this talk

1. 1:41

   Why contact matters

   Chong Tong argues that policies trained only to predict kinematic targets can miss the forces and contacts that determine whether a task succeeds. In wiping and box-flipping examples, baseline policies apply too much force or fail to apply enough. The project targets contact-rich tasks such as jar opening, wiping, flipping, and grasping fragile objects.

2. 6:39

   A measurable representation of contact

   The team represents contact using three signals: tactile sensing, the robot’s actual state, and the controller’s target. With a compliant controller, the gap between target and state provides a force proxy, while a force-torque sensor can help estimate the contact patch. In simulation, a small network trained on two signals predicted the third, including for grasps absent from its training data.

3. 14:26

   Predicting contact the robot can execute

   The policy uses diffusion to predict tactile signals and robot state, then maps those predictions to a control target that a compliant controller can execute. The mapping is intended to ensure that predicted contact is physically achievable. Tactile images are compressed into a small latent space; the system uses six cameras and runs at about five hertz.

4. 21:26

   Teleoperation and task results

   The team collected demonstrations using motion capture in the real world and a Meta Quest in simulation, retargeting human fingertips to the robot and using compliant control. They report that Contact-Grounded Policy performed best across five tasks compared with visuomotor and tactile-conditioned diffusion policy baselines, with about a 15% performance improvement. Demonstrations include smoother dish wiping, box flipping that requires enough force, and jar opening.

5. 29:09

   Limits and future directions

   The current contact mapping assumes fixed controller gains and rigid objects. Tong says future versions could condition on gains, controller type, robot dynamics, and object properties; visual and tactile history may help infer softness. He also sees potential in mapping glove-recorded hand poses to the control targets needed for physical interaction, and in testing whether the contact mapping can transfer across tasks.