"Scalable Robot Learning in Rich Environments" - Raia Hadsell

In this talk

1. 1:43

   Why richer environments matter

   Hadsell argues that when an agent can model its whole environment, it can succeed by memorizing rather than learning transferable skills. She says environments become more challenging through noisy real-world data, limited views, multiple senses, interactions, multiple agents, and persistent changes. That pressure can encourage generalization, adaptation, continual learning, and causal reasoning.

2. 11:43

   Learning reusable movement skills

   Hadsell describes using motion capture to train a humanoid controller that can serve as a reusable movement module for later tasks. In COMIC, the agent learns to imitate motion clips and solve tasks at the same time, using separate imitation and task rewards. An MLP transferred better across several tests than an LSTM, although the resulting controller was less capable on parkour than one trained end to end; it trained in minutes rather than days.

3. 27:03

   Imitating manipulation from video

   The second project aims to let a robot follow a manipulation demonstration filmed from another viewpoint, without task rewards, robot actions, or proprioception in the demonstration. Hadsell’s team trained a visual representation using simulated trajectories with randomized environments and arms, including invisible arms. The representation supported transfer between a human hand and robot arms, and performed best on lifting and stacking tests among the representations compared.

4. 41:29

   Discussion: rewards and broader data

   In discussion, Hadsell says task-specific reward shaping can produce about 75% stacking success in simulation, but requires rewards for subtasks and conditions. The video imitation method reached about half that success without a customized reward or task-specific training. She also points to YouTube as a possible source of broad robotics data, while noting that learning robot behavior from it remains difficult.