David Held, assistant professor at Carnegie Mellon University's Robotics Institute, discusses designing robot policies that adapt quickly after deployment at an IEEE Robotics and Automation Society event.
Held contrasts robots that perform impressive demonstrations with robots that can reliably handle everyday manipulation, such as cooking. Large datasets and pretrained knowledge can help a policy understand the world, but may not give it dexterity. Training and deployment differ, and hidden object dynamics can cause failures that require learning from interaction.
For opening unfamiliar doors and drawers, Held's team generated more than 40,000 simulated demonstrations. Their hierarchical imitation method predicts useful gripper poses as subgoals, then uses a low-level policy to carry them out. It worked across three robot embodiments and outperformed an end-to-end policy trained on the same data; a later multitask version used over 180,000 trajectories for opening, closing, grasping, and placing.
The multitask policy served as a foundation for new tasks: with three demonstrations, it learned laundry folding and achieved 60% performance after five folds, compared with about two successful folds for Pi-Zero-5. For dexterous manipulation, the team used diverse grasp types from Dexonomy and fine-tuned on tasks with 50 demonstrations. The results point to a goal-conditioned low-level policy as a key source of transfer.
Held argues that a scalar reward hides useful information: when an off-center bar tilts after a lift, the direction of the tilt suggests how to adjust the grasp. His team's meta-learning method updates a latent belief about object dynamics using prediction errors from interaction history. On challenging ambiguous-object tasks, it achieved a 3% error rate, and one pick or push could reveal the bar's center of mass.
For unfamiliar tools, Held describes generating several candidate plans with a vision-language model, then using a world model to predict their outcomes. Because both the plans and model can be wrong, the robot executes actions, updates its model, and uses it to choose better plans. Held identifies human feedback as another opportunity for improving fast adaptation.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
David Held, faculty at Carnegie Mellon University, presents research on precise, generalizable robot manipulation at the 2026 Spring Robotics Colloquium, hosted by the Paul G. Allen School and introduced by Abhishek Gupta.
David Held, associate professor at Carnegie Mellon University and head of the RPAD Lab, presents research on precise, generalizable robot manipulation at a GRASP on Robotics seminar hosted by a Penn faculty member.
David Held, a Carnegie Mellon University faculty member, presents research on perceptual robot learning at the UW Robotics Colloquium, introduced by Dieter Fox.
David Held, an assistant professor at Carnegie Mellon University's Robotics Institute, discusses perceptual robot learning and manipulation research with the umrobotics audience.
