Shuran Song, a robotics researcher at Columbia and now Stanford, presents at the CMU Robotics Institute seminar on learning to use real-world dynamics in robot manipulation.
Song contrasts a slow warehouse robot with workers who rapidly toss and sort packages. Static manipulation assumes objects stop when the robot stops; dynamic manipulation uses forces and acceleration to move objects faster or extend a robot’s reach. She identifies modeling the external world, including unfamiliar objects and their physical properties, as a central challenge.
FlingBot learns grasp locations from camera images and uses a fixed, hand-coded fling action to unfold cloth. Self-supervised rewards and automated resets help the robot collect data, while spatial transformations improve learning efficiency. It reached over 80% cloth coverage within three interactions and generalized to unseen shapes, but struggled with cloths whose weight or length exceeded its capabilities.
For precise tasks such as swinging a rope toward a target, Song describes Iterative Residual Policy, which predicts how changes to an action will change an observed trajectory. Its model is trained in simulation despite a substantial sim-to-real gap, then used to adapt actions across ropes with different properties. After knots changed a rope’s length and density, the system’s error rose before it adapted over several iterations.
Song presents interactive perception as a way to learn about objects through manipulation. In the DynaMet project, sliding and collision actions help a visual prediction model distinguish objects with different physical properties; slow pushing alone provides too little evidence. The decoded mass and friction values are approximate, and serve mainly to assess whether the learned representations capture useful differences.
Song describes manually designed action primitives as useful for learning with limited data, but costly to engineer and limited in scope. She presents diffusion policies trained on teleoperated demonstrations as a more general skill representation, showing a robot that combines picking, placing, and pushing to flip a mug. She closes by arguing that robots can use dynamics, including acceleration and airflow, for both manipulation and gathering information.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
Stanford University Professor Shuran Song explains how to teach robots manipulation skills through natural human demonstrations at an IEEE Robotics and Automation Society talk.
Shuran Song, a robotics researcher at Columbia University, presents her work on dynamic manipulation of deformable objects at the GRASP Lab Spring 2022 SFI.
