David Held, associate professor at Carnegie Mellon University and head of the RPAD Lab, presents research on precise, generalizable robot manipulation at a GRASP on Robotics seminar hosted by a Penn faculty member.
Held contrasts broad vision-language-action models, which can identify a requested target but may place an object inaccurately, with specialist systems that perform narrow tasks well. He argues that robots need both broad task understanding and the precision of specialized methods, and proposes a hierarchy that separates choosing a goal from executing motion.
Held describes training on more than 40,000 simulated trajectories for opening doors, drawers, and other articulated objects. An end-to-end diffusion policy struggled with unseen objects, while a hierarchical imitation approach predicted key gripper poses and used a lower-level policy to produce motion. The resulting policy transferred from simulation to three robot embodiments in real settings.
The SMITH follow-up applies hierarchical policies to opening, closing, grasping, and pick-and-place, using 140,000 simulated demonstrations. Its high-level dense Gaussian mixture model represents multiple possible target poses, supporting multimodal tasks such as grasping in clutter. Held reports better performance than the compared generalist models and results close to task-specific specialists, while noting the system uses a short list of fixed language prompts.
For tasks ranging from mug hanging to industrial insertion, Held describes diffusing object point clouds toward a target placement and then using a planner or learned policy to execute the motion. The method separately predicts object shape and placement frame, and initializes the frame estimate with a dense Gaussian mixture model. In tests on three NIST insertion connectors, reported success ranged from 80% to 100% at roughly millimeter and one-degree precision; point-cloud diffusion also generalized better than SE(3) diffusion across diverse mugs.
Held summarizes the shared structure: a high-level model predicts a goal or placement, and a low-level component handles motion. He identifies key-frame selection, combining tasks without negative transfer, and scaling to more tasks as unresolved challenges. He also points to tool use and learning with planning as future directions.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
David Held, faculty at Carnegie Mellon University, presents research on precise, generalizable robot manipulation at the 2026 Spring Robotics Colloquium, hosted by the Paul G. Allen School and introduced by Abhishek Gupta.
David Held, a Carnegie Mellon University faculty member, presents research on perceptual robot learning at the UW Robotics Colloquium, introduced by Dieter Fox.
David Held, an assistant professor at Carnegie Mellon University's Robotics Institute, discusses perceptual robot learning and manipulation research with the umrobotics audience.
