RI Seminar: Oliver Kroemer : Learning Robot Manipulation Skills...

In this talk

1. 1:37

   Learn task structure from robot experience

   Kroemer frames manipulation around efficient learning, robustness, and versatility in uncertain environments. He describes a model that identifies distinct interaction modes, such as approaching an object, making contact, and moving it, and predicts when transitions between modes are likely.

2. 14:40

   Build reusable skills and sequence them

   The method segments demonstrations into modes, learns a motor skill for each transition, and optimizes those skills with reinforcement learning. In a two-handed pickup, optimization improved lifting success by strengthening the grasp. A high-level policy then chose and ordered skills to move a box toward different goals.

3. 25:56

   Monitor outcomes and recover from errors

   For a peg guided through a groove into a hole, tactile signals and classifiers helped detect whether each motion reached its goal; the robot retried when it fell short. In scooping experiments, vibration from a contact microphone helped estimate material quantity. For coffee beans, shaking-based estimates had about five grams of RMS error.

4. 38:08

   Transfer relevant features across tasks

   Kroemer describes selecting object features that matter to a skill and using information from earlier tasks to guide that selection. A prior learned from pushing helped with placing, where the tasks shared alignment cues, but transferred less effectively to tilting. It also helped a pouring skill account for alignment and a wiping skill adapt to the surface position.

5. 52:26

   Toward lifelong manipulation learning

   Kroemer summarizes the work as combining representations of tasks, actions, and objects to support efficient, robust, and generalizable manipulation. He points to agriculture, cooking, aerial manipulation, and industrial work as areas for future skills.