Matej Hoffman - Humanoid and Cognitive Robotics

In this talk

1. 0:50

   Whole-body skin for safer interaction

   Hoffmann describes whole-body tactile skin as a way to detect, locate, and respond to contact, supporting both safety and social interaction. In tests on three robots, including an industrial robot and two collaborative robots, skin reduced collision forces across about 2,000 measurements. Lower forces can allow robots to move faster while staying within safety limits.

2. 3:17

   Touch and vision guide robot motion

   On iCub, which has about 4,000 tactile receptors, Hoffmann demonstrates a controller that handles a person pushing the robot during a card game. It tracks tactile and visual obstacles across the body and feeds them into a controller for 17 degrees of freedom. The controller updates constraints every 10 milliseconds while keeping motions smooth.

3. 5:18

   Robots calibrate themselves through touch

   Hoffmann compares traditional calibration, which needs external equipment or fixtures, with self-calibration using a robot’s own sensors. In self-touch, the robot compares the estimated positions of two fingertips; in self-observation, it compares an expected finger position with its camera view. Self-touch proved robust, and combining it with self-observation improved results.

4. 8:12

   Learning body maps and object properties

   Hoffmann shows an iCub tactile map trained by moving a thumb over its skin for about half an hour, and connects this work to models of infant sensorimotor development. In grasping experiments with four hands or grippers, the gripper and action parameters mattered more for object discrimination than the classifier or feature preprocessing. Different speeds and grasp configurations changed how well objects could be distinguished.

5. 12:50

   Why haptic foundation models are difficult

   Hoffmann argues that cognition depends on interactions among brain, body, and environment, while many foundation models learn from passive data and operate mainly in an outer control loop. Haptic sensing is active, and results depend strongly on hand design, sensor placement, and grasp actions. He also notes that robot joint encoders provide accurate joint states, unlike human proprioception, making a comparable proprioceptive body map harder to define.