Ep#104: Light-Loco-Parkour: Versatile Perceptive Whole-Body Locomotion via Multi-Skill Distillation

In this talk

1. 1:15

   One policy for walking and parkour

   The 90-centimeter robot uses onboard sensors to walk across planks and stepping stones, handle stairs and slopes, and perform vaults and climbs. It can scale boxes about 85% of its height; some climbs use its arms and torso as well as its legs.

2. 2:57

   Building and expanding parkour references

   The team identifies scarce, difficult-to-retarget human motion references, plus skill choice and smooth transitions, as key challenges. They derive motions from online videos, align obstacles and contact points by hand, then use simulated rollouts to raise obstacle height in 5 to 10 centimeter steps. Human judgment still guides reference selection and contact-link choices.

3. 6:47

   Training teachers and learning transitions

   Separate locomotion and parkour teachers are distilled into one policy, then trained to switch skills on a course with known sections. Transition training substantially improves simulated success: without it, switching to the correct behavior rarely succeeds, while the transition-trained policy can be deployed on the robot.

4. 22:42

   Deploying with depth and memory

   The team distills the policy to use an onboard depth camera and adds a GRU to retain information when the camera cannot see beneath the robot or near its feet. Training includes depth noise, missing image regions, and latency. Without the GRU, tests on stepping stones fail; further fine-tuning also improves some obstacle traversals.

5. 27:47

   Limits and possible extensions

   Preparing references and adding skills still require substantial human work, and the project demonstrates three parkour skills. The team says camera movement or a wider field of view could improve perception. They have also applied the pipeline to carrying and placing boxes, while noting that planning object-dependent skill sequences remains an open challenge.