Ep#7 AnyDexGrasp: Learning General Dexterous Grasping for Different Hands

In this talk

1. 2:20

   One grasping approach for different hands

   Hu describes a way to adapt grasping to hands with different shapes and finger counts without building a large labeled dataset for every hand. The method first learns contact locations and surface directions from GraspNet-1Billion, then uses a smaller set of real grasp attempts to adapt to a particular hand. He reports that collecting those attempts takes about four to eight hours per hand.

2. 6:40

   A compact representation cuts the learning burden

   The system turns a single-view point cloud into candidate contact positions and surface normals, then classifies hand configurations based on whether real attempts succeed. Hu says each local grasp is encoded with 480 values, making the classifier small and fast. Rather than optimize a grasp pose, the system samples thousands of wrist poses and hand configurations.

3. 15:57

   Dense sampling supports generalization

   Training used 40 objects, while tests included more than 100 unseen objects, including deformable items and challenging examples from YCB and DexNet. Hu says performance on deformable objects was strong, possibly because they conform to the hand. The hosts and Hu explain that sampling local surface geometry densely can matter more than adding many objects: the dataset included over 20 million sampled poses per training object.

4. 5:21

   More fingers need more feedback

   With the training sample count held constant, success was highest for the three-finger hand at roughly 90 to 97%, falling to about 85% for a five-finger hand and 75 to 80% for the 16-degree-of-freedom Allegro. Hu suggests hands with more joints need tactile sensing because their movements create and break more contacts. The method is open loop, so it cannot adjust if an object shifts during the grasp.

5. 23:39

   Useful grasps depend on the task

   The researchers divide hand postures into a limited set of grasp types, drawing on human grasp taxonomies. A single posture with the highest success rate did not perform best in every situation, partly because other postures can avoid collisions. Hu says dexterous hands can handle a wider range of tasks and may complete some tasks faster, but this grasping system needs other modules such as segmentation and planning.