Fall 2025 GRASP SFI - Huy Ha, Columbia University & Stanford University

In this talk

1. 0:59

   Make robot building community driven

   Ha contrasts systems designed around an expert’s chosen task with an approach that lets more people shape robot behavior. He argues that human images, language, and movement can each inform different parts of a robot, widening who can contribute.

2. 3:37

   Collect demonstrations without a robot

   UMI is a low-cost camera-equipped gripper people can use to record manipulation examples; Ha says roughly 100 units could be made for the price of one UR5e. Each recording can be processed into gripper motion and visual input for training a policy, including for dynamic tasks such as tossing.

3. 5:36

   Train a controller for legged manipulation

   Ha describes a whole-body controller trained with reinforcement learning in simulation, without simulating the demonstrated tasks. It receives a sequence of end-effector targets in world coordinates, so it can anticipate movement and compensate for the quadruped’s body motion. An iPhone mounted on the robot supplies real-time pose estimates for stabilization outdoors.

4. 16:24

   Test performance and transfer

   The system achieved a 70% success rate on targeted tossing and 90% on autonomous pushing; heavy-object trials exposed overheating and power limits. A separately developed cup-handling policy also ran on the quadruped, reaching 60% success across four cups and five outdoor locations. Ha says removing future targets, world-frame tracking, or other controller components hurt tossing performance.

5. 26:38

   Build a shared robotics ecosystem

   Ha highlights that a visuomotor policy trained from another group’s data could run on both a fixed-base arm and a quadruped with different controllers. He describes UMI on Air, where demonstrations gathered at Stanford were sent to a Carnegie Mellon team in Pittsburgh for drone deployment. He also reports that UMI Data added 40,000 demonstrations in the previous 12 months.