Spring 2026 GRASP Seminar - Glen Berseth, Université de Montréal

In this talk

1. 1:19

   Why robots still need to adapt

   Berseth describes robotics as an old field whose capabilities are changing with recent advances in AI. He focuses on applications such as manufacturing, assembly, recycling, and agriculture, and frames the central challenge as helping agents learn from new situations and use that knowledge effectively.

2. 6:19

   Object segmentation for visual reinforcement learning

   Berseth presents a method that uses text-guided object detection and a faster Segment Anything variant to turn camera images into per-object embeddings for reinforcement learning. In tests including pushing and pulling cubes and moving fruit into a basket, the approach trained in about 20 hours and retained performance better under environment changes than several comparison methods. He notes that the method still needs evaluation on more complex tasks and scenes.

3. 20:06

   Using a VLA to guide model-based search

   Berseth describes using a pretrained vision-language-action model to propose action chunks, then searching locally around those proposals with Monte Carlo tree search. The search improved performance on LIBERO tasks, but could take several minutes per trajectory when the policy was weak. The current experiments use the simulator as the world model; replacing it with a learned model remains difficult, in part because available demonstrations contain few failures.

4. 32:31

   More inference for better language model answers

   Berseth describes a recursive sampling method that generates candidate answers, selects a promising group, and asks the same language model to produce improved candidates. A few rounds generally helped across reasoning tasks, while additional rounds offered little benefit; the approach also consumed substantial inference compute. Reinforcement learning fine-tuning of the aggregation step provided a further performance gain.

5. 42:30

   Reinforcement learning and optimization limits

   Berseth examines why policies may fail to learn from strong trajectories already present in their own generated data, using deterministic Atari environments to reduce randomness as an explanation. He reports that PPO tracked changing data better than DQN in his experiments, and that K-FAC, a second-order optimizer, worked best among the methods he tried. He sees optimization under changing data as a key open problem for both robotics and language model fine-tuning.