Shuran Song, a robotics researcher at Columbia University, presents her work on dynamic manipulation of deformable objects at the GRASP Lab Spring 2022 SFI.
Song contrasts slow, careful robot movements with the faster actions people use to handle objects. Cloth is difficult because it has many degrees of freedom, and the robot directly controls only the parts it holds. Using motion and momentum can help manipulate those out-of-reach areas more efficiently.
FlingBot learns where to grasp cloth, then uses a fixed sequence to stretch, swing, and release it. Two UR5 arms and a top-down camera provide the actions and coverage feedback, allowing self-supervised training without labeled examples. Song reports over 80% coverage on most tested garments within three interactions, and over 90% after four.
Song describes using a blower to move deformable objects without direct contact or fast robot motion. A camera observes the cloth so the system can adjust the blower direction, including for large garments that FlingBot could not fully unfold. For bag opening, the team collected 4,000 real-world blowing sequences over ten hours and trained a policy that generalized from one training bag to bags with different sizes and materials.
For tasks that require a specific result, such as swinging a rope toward a target, Song’s team predicts how small action changes will alter the observed trajectory. The robot tests candidate adjustments, observes the outcome, and iterates; rope trials typically converge in about eight or nine steps. The same approach was applied to cloth placement, where trials usually took three steps, and it adapted to different ropes and robot configurations.
Song identifies richer visual state estimates and automatic discovery of useful motion strategies as important next steps. She notes that the systems rely on manually designed action primitives and that some trial-and-error methods require repeatable tasks. Learning from human demonstrations could help robots find useful actions beyond those engineers specify.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
Stanford University Professor Shuran Song explains how to teach robots manipulation skills through natural human demonstrations at an IEEE Robotics and Automation Society talk.
Shuran Song, a robotics researcher at Columbia and now Stanford, presents at the CMU Robotics Institute seminar on learning to use real-world dynamics in robot manipulation.
