GRASP On Robotics: Byron Boots - April 9th, 2021

In this talk

1. 3:10

   Why online learning fits robotics

   Boots frames robot learning as a combination of policies, objectives, and algorithms that must handle changing data and imperfect models. Online learning offers a way to update decisions as a robot interacts with its environment, without relying on batch data or independent samples.

2. 25:42

   Using simulation with care

   Simulation can make policy updates cheaper, but it only approximates what happens on a real robot. Boots describes the gap between simulated and real performance and suggests using simulation to make early progress, followed by real-world refinement.

3. 31:25

   Deriving MPC from online learning

   Boots presents model predictive control as repeatedly optimizing a simple sequence of actions using a simulator, then executing the first action and shifting the sequence forward. On off-road racing trucks, this approach achieved lap times about 10% faster than human pilots; tuning the update step also cut the rollout count from nearly 2,000 trajectories to 64.

4. 46:50

   Applying MPC across terrain

   At the University of Washington, Boots's team uses MPC within a larger perception and mapping system on a Clearpath Warthog. The robot handled dirt, snow, and slush without changing the controller or its dynamics models, illustrating how local replanning can help with varying conditions.

5. 50:46

   Combining MPC and reinforcement learning

   Boots describes ways to help MPC improve with experience, including learning a Q-function from collected robot data and using it to guide simulated rollouts. For legged robots, his group also explores having reinforcement learning choose objectives for a lower-level MPC controller instead of directly selecting motor torques.