David Held, an assistant professor at Carnegie Mellon University's Robotics Institute, discusses perceptual robot learning and manipulation research with the umrobotics audience.
Held argues that robots need representations shaped by what they must do with objects, not only by what images contain. For example, recognizing a towel is different from predicting how it will respond to a robot's actions. His research focuses on learning to manipulate unfamiliar, deformable objects.
Held introduces SoftGym, a benchmark built on NVIDIA Flex with tasks such as cloth folding, rope straightening, and moving water without spills. In benchmark comparisons, leading reinforcement learning methods lagged behind an oracle using ground-truth state and dynamics, especially for cloth spreading and folding. He shows how latent models can miss small but important details, such as wrinkles or water droplets.
For cloth smoothing, Held's team represents visible cloth points as a graph and predicts their motion with a graph neural network. Trained in simulation on 2,000 random pick-and-place actions, the model transferred to cotton and silk cloth and a T-shirt. It outperformed pixel dynamics, latent dynamics, and a model-free policy, and showed transfer to cloth shapes absent from training.
A second project uses flow to estimate how points in the current cloth observation should move to match a goal. The system uses those correspondences to choose grasp and placement points, select single-arm or bimanual actions, and decide when to move to the next subgoal. Trained in simulation, it performed real-world folding on rectangular cloth and T-shirts.
Held connects the two projects through their focus on relationships: between parts of an object and between its current state and a goal. He identifies future directions including clothing, dough, granular materials, mixed materials, object interactions, and longer-horizon manipulation.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
David Held, faculty at Carnegie Mellon University, presents research on precise, generalizable robot manipulation at the 2026 Spring Robotics Colloquium, hosted by the Paul G. Allen School and introduced by Abhishek Gupta.
David Held, associate professor at Carnegie Mellon University and head of the RPAD Lab, presents research on precise, generalizable robot manipulation at a GRASP on Robotics seminar hosted by a Penn faculty member.
David Held, a Carnegie Mellon University faculty member, presents research on perceptual robot learning at the UW Robotics Colloquium, introduced by Dieter Fox.
