Fall 2022 GRASP SFI - Zhongyu Li, University of California Berkeley
Zhongyu Li, a PhD student at Tsinghua University working with UC Berkeley’s Koushil Sreenath, presents research on combining model-based safety methods with model-free reinforcement learning for legged robots.
In this talk
- 0:30
Why legged robot control is difficult
Li describes Cassie as a 20 degree-of-freedom biped with nonlinear, hybrid dynamics and passive spring joints. Full robot models are complex, so model-based control needs careful modeling and tuning, while model-free reinforcement learning can produce agile behavior but lacks formal safety guarantees.
- 5:14
Model-based methods for navigation and safety
Li gives examples of trajectory optimization, reduced spring-loaded inverted pendulum models for obstacle avoidance, and control barrier functions combined with model predictive control. Robots used these methods to pass through tight spaces, guide a person by leash, and coordinate around obstacles while pulling a load.
- 11:38
Learning robust locomotion across robots
For Cassie, a neural network policy tracks reference motions using state and action histories, with simulation randomization to help transfer to hardware. Li reports recovery from disturbances and control across varied speeds and heights. A separate policy trained with randomized quadruped morphologies transferred to robots including A1, Mini Cheetah, AlienGo, and Spot.
- 26:39
Combining learned control with identified models
Li treats the learned controller and robot as a closed-loop system, then fits linear input-output models from command and motion data. At low input frequencies, the tested dimensions were approximately linear and decoupled; the models had stable, minimum-phase poles and zeros. Using the identified model in online navigation let Cassie avoid obstacles and finish a demonstrated task twice as fast as the earlier model-based method.
- 36:26
Limits, open questions, and deployment
Li says model-free policies can reduce robot-specific control design and tuning, but their behavior remains difficult to explain or formally guarantee. The identified models also have fitting errors, and the talk notes that robust control barrier functions for MPC remain an open problem. In the Q&A, Li says system identification was performed offline and that online identification is a direction for future work.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.