"Handy Moves: Dexterity in Multi-Fingered Hands", ICRA 2025 Workshop

In this talk

1. 8:39

   Cross-embodiment learning

   Shuran Song presents two ways to transfer manipulation across human and robot hands. DexUMI uses a hand-specific exoskeleton to constrain demonstrations to a robot’s capabilities, record contact through direct physical interaction, and adapt camera views for policy training. Her Graph Embodiment Transformer represents each hand’s joints and links as a graph, aiming to let one policy work across different hand designs.

2. 34:50

   Simulation and human demonstrations

   Ankur Handa describes reinforcement learning in GPU simulation for dexterous manipulation, including cube rotation on a LEAP Hand using vision without tactile sensors or cube markers. Vectorized simulation cut training from months to about two and a half days. Tony Liu presents a complementary data approach: motion-capture demonstrations, a reusable hand imitation policy, and a residual policy that accounts for object interaction; the resulting DexMan dataset contains 3,300 trajectories across 61 tasks.

3. 1:33:41

   Hands that use touch and compliance

   A. Miller argues that manipulation capability can come from how a hand applies force and exploits contact, rather than from hand complexity alone. His examples include retrieving feasible grasps from a precomputed library, using tactile sensing to detect slip, and planning sequences for holding several objects. Later speakers discuss soft hands and distributed stiffness, showing how compliance and the surrounding environment can help shape grasps and support manipulation.

4. 1:53:05

   Data, policies, and embodiment

   Larry Pinto demonstrates VR-based teleoperation and learning for tasks such as in-hand object rotation, retrieving an object from inside a larger one, and cutting wire with a tool. He emphasizes tactile feedback for contact-sensitive tasks and describes KUDOR, which learns from a human video with online robot training. Another speaker describes aligning human and humanoid actions in human hand-pose space; combining varied human demonstrations with robot data improved generalization to new objects, locations, and backgrounds.

5. 2:17:45

   Research priorities and practical limits

   In the panels, researchers identify realistic simulation, robust hardware, repairability, affordable full-hand tactile sensing, and better evaluation as open challenges. Oliver Brock proposes organizing manipulation around reusable primitives and compliance, with plans instantiated using information available during execution. Industry and academic speakers also discuss co-designing hands and controllers, while noting that multifingered systems remain far from affordable, reliable commercial performance for many tasks.