2024 Winter Robotics Colloquium: Matei Ciocarlie (Columbia University)

In this talk

1. 3:24

   From grasping to dexterous manipulation

   Ciocarlie looks back at Willow Garage work, where robots used vision and motion planning or compliant grippers to pick up objects. He argues that grasping familiar, reachable objects was largely addressed under those conditions, while manipulating an object after grasping it remained a harder challenge.

2. 8:22

   Tactile sensing for active exploration

   Ciocarlie describes the Disco Finger, built with John Kymissis, to provide broad tactile coverage over curved surfaces. Its roughly 1,000 raw signals are difficult to interpret analytically, so supervised learning maps them to contact location and force. He also shows tactile object recognition, where an explorer chooses where to touch to help a classifier identify an object.

3. 17:44

   Planning exploration for in-hand reorientation

   The lab’s goal was to rotate objects in the hand without vision or a support surface. Ciocarlie explains how sampling based motion planning helps explore the 21 dimensional state space of a 15 joint hand and a six degree of freedom object, while maintaining stable contact. The resulting paths seed reinforcement learning, which learns finger gaiting and transfers to a real hand using domain randomization.

4. 38:32

   Exploration remains a central challenge

   Ciocarlie says dexterous manipulation has advanced, but success still depends on a task the hardware can perform, a workable simulation, and substantial training and tuning. He identifies exploration, especially learning when and how to move a finger off and back onto an object, as a major difficulty. Human demonstrations can help by guiding learning through challenging states.

5. 46:26

   Co-designing hardware and learning

   Ciocarlie argues that hardware design and learning should be optimized together. His examples include using reinforcement learning to optimize hand transmissions and a two-level method that pairs a differentiable neural proxy with a physics based hardware model. He sees potential to apply co-design to robot kinematics, actuators, and sensor placement, and also describes a hand orthosis project for stroke rehabilitation.