Dynamic Robot Manipulation: Learned Optimization, Deformable Materials, and the Cloud

In this talk

1. 4:19

   Planning faster pick-and-place motions

   Ichnowski combines grasp choice, obstacle avoidance, and motion timing in one trajectory optimization. On a real robot, a learned trajectory gives the optimizer a warm start, reducing computation from 28 seconds to 80 milliseconds while preserving constraint checks. He also uses reinforcement learning to tune a parameter in the OSQP quadratic-program solver, reporting faster results across several benchmark problems.

2. 21:05

   Accounting for motion and deformable objects

   For open containers, the planner constrains acceleration so carried contents are less likely to spill; the resulting motions can include swinging the container or turning it upside down. For suction transport, a neural network learns from pressure-sensor data and acceleration histories when a grasp is likely to fail, enabling faster moves than the analytic comparison model. Cable experiments use a reset motion to make starting conditions repeatable, then learn trajectories for moving cords around obstacles.

3. 36:56

   Using cloud resources for robot planning

   Ichnowski describes lock-free sampling-based planning that can use many CPU cores at once, including a whole-body plan for putting a book in a drawer. He also presents a planner distributed across serverless computing instances, where sharing the best path helps reject samples unlikely to improve it. FogROS 2 is intended to simplify running robot workloads on cloud hardware; examples include a 12-fold speedup for GPU-based grasp planning and up to 28-fold for multicore motion planning.

4. 44:42

   Limits and next steps

   Cloud execution raises timing, safety, privacy, and security questions that the team is still addressing. Ichnowski points to future work on distributing computation with awareness of critical tasks and privacy, and on adding features such as multi-robot coordination. He also notes that fast transport plans need better estimates of spill thresholds and grasp resistance.