2024 Winter Robotics Colloquium: Matt Barnes (Google Research)
Matt Barnes of Google Research explains how inverse reinforcement learning can use real user routes to improve Google Maps, presenting at the Paul G. Allen School Robotics Colloquium at the University of Washington, where he was introduced by Siddhartha Srinivasa.
In this talk
- 3:14
Routing as inverse reinforcement learning
Barnes frames route choice as an inverse RL problem: road segments are states, turns are actions, and trips from Maps users are demonstrations. Learning preferences from those trips could help account for factors such as safety and road comfort, which are harder to measure than driving time and matter especially for walking and cycling.
- 13:48
Scaling to a global road network
The system covers hundreds of millions of road states and demonstrations. Barnes describes compressing the graph to reduce wasted memory, cutting training time by nearly three times, and splitting the world into about 250 geographic regions so each could be trained as a local model.
- 18:48
Faster methods and route results
MaxEnt++ starts its calculations from shortest-path costs, reducing training time by about 16% without lowering reported metrics. Receding Horizon Inverse Planning combines stochastic planning near a demonstrated route with a cheaper deterministic planner farther away. Together with the other system components, the approach improved route conformance by 16% for driving and 24% for two-wheelers against the production baseline.
- 32:54
Practical lessons from deployment work
Visualizing model behavior helped Barnes trace sudden training loss spikes to policies looping through parking lots, where many short paths could destabilize MaxEnt. A sparse reward model also surfaced map errors, including a Nottingham road incorrectly marked private. Barnes highlights data quality, mismatches between offline training and online conditions, noisy user demonstrations, and the benefits of learning rewards that can be reused across destinations.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.