2024 Fall Robotics Colloquium: Kiana Ehsani (PRIOR and the Allen Institute for AI)

In this talk

1. 1:28

   What a robotics breakthrough would require

   Ehsani compares expectations for robotics with ChatGPT’s broad task coverage, ability to handle unfamiliar situations, and adaptability. She says robotics still falls short, and frames her presentation around data, pretraining, and adaptation.

2. 5:57

   Learning navigation from simulation

   Her team trained a transformer policy on diverse simulated shortest-path demonstrations, using frozen image and text encoders and extensive data augmentation. The policy learned behaviors such as searching for the highest fruit in a kitchen and transferred to real environments. Ehsani notes that this approach benefits from tasks where information-gathering actions are less central.

3. 13:09

   Combining imitation learning with reinforcement learning

   Ehsani describes imitation learning as efficient but limited by the states its demonstrations cover, while reinforcement learning can improve behavior through trial and error but is costly and unstable. Her team sped up navigation training with parallel rollouts and transformer caching, then fine-tuned pretrained policies using sparse success rewards. On one task, success rose from 20% to 70% after 50 million frames; a lower learning rate, no entropy bonus, and separate actor and critic networks helped stabilize training.

4. 27:00

   Adapting policies across tasks and robots

   Fine-tuning helped policies handle related tasks without expert demonstrations in the new domain, and adapt from Stretch to LoCoBot. In another project, the team randomized simulated robot bodies and camera parameters to train navigation policies across different embodiments. Ehsani says these results are promising but preliminary, and the evaluation shown focused on navigation.

5. 41:32

   Using Molmo to ground robot instructions

   Ehsani argues that simulation cannot cover every real-world object and semantic detail, such as identifying a colleague’s desk or choosing the largest cup. She presents Molmo as a vision-language model that can identify points in images for a low-level controller to use, and describes data collected from 400,000 images, producing 2.3 million question pairs. She identifies complex manipulation, faster inference, error recovery, and adding depth or proprioception as open challenges.