[DR02 Pro][Locomotion] [RL] DR02 humanoid robot locomotion, open-source rl training code explanation

In this talk

1. 1:23

   Codebase and robot model

   The DR02 training project has setup modules for actuators, assets, environments, managers, and tasks, plus an RLSRL module implementing PPO and AMP. Training processes the robot’s URDF by fixing selected joints, and a randomized delayed PD actuator varies motor strength, control parameters, position bias, and joint torque-speed behavior.

2. 5:23

   Environment and training pipeline

   The generic AMP environment config defines commands, actions, observations, rewards, randomization events, and termination rules; DR02-specific settings such as joint ordering and action scales are in a separate config. Policy inputs use deployable proprioception, while the critic and AMP discriminator receive additional information. Training runs at 50 Hz with four 5 ms physics steps per environment step, and scheduled pushes begin after the velocity reward reaches 93%.

3. 21:45

   PPO, estimation, and AMP

   The custom PPO runner combines policy learning with an explicit-implicit estimation network and an AMP discriminator. The actor uses proprioceptive observations, estimated base velocity, and a latent representation; a decoder learns observation features, while the critic uses privileged inputs. Separate optimizers handle PPO, discriminator, and estimation losses, and symmetry augmentation and mirror loss encourage consistent left-right motion.

4. 32:14

   Symmetry and deployment

   Symmetry training mirrors observations and compares the resulting actions with mirrored original actions. Because the actor also needs an encoded latent, the system derives a matching latent from mirrored observation history. For deployment, the actor and encoder are exported together; the SDK policy runs at 50 Hz while its state machine runs at 500 Hz, so deployment decimation is 10.