Sim2Real
A Montreal Robotics instructor discusses how reinforcement learning and simulator design can help transfer robot policies to real hardware, using Cassie as a detailed example.
In this talk
- 3:14
Why train policies in simulation
Collecting robot data is slow: an arm action can take about two seconds, and building large robot farms still falls short of simulation’s speed. The instructor contrasts that with GPU simulation, which can train a policy in 10 to 15 minutes. Simulators also provide reliable rewards and make it easier to debug policies before hardware trials.
- 10:30
Where simulators fall short
Fast robotics simulators simplify physics and struggle with soft objects, complex contacts, and friction. Sim-to-real has worked especially well for walking and other tasks with simple geometry and limited object interaction. Manipulation is harder because useful environments need varied objects that move and can be interacted with, such as doors, appliances, and their contents.
- 51:24
Randomize dynamics to narrow the gap
The real robot may experience different dynamics from the simulator, so training should vary parameters such as friction, mass, and ground properties. The policy learns to work across those settings without being told which one is active. This takes more data than calibrating a specific robot through system identification, but can make the policy robust to uncertain hardware conditions.
- 1:04:39
Cassie’s transfer process
For Cassie, the team trained in MuJoCo, then checked policies in a MATLAB based SimMechanics model that could represent the robot’s springs before testing on the physical robot. A policy used several recent steps to infer the robot’s current dynamics and produced joint position commands at roughly 30 to 60 hertz, while the robot’s lower level control ran around 500 hertz. The team randomized mass, friction, sensor noise, and communication delay, then adjusted rewards after real-world tests exposed hard impacts and broken feet.
- 47:36
Use simulation as a starting point
Simulation supports fast reward design, policy debugging, and pretraining, but most strong transfer workflows still include some real-world data collection or fine-tuning. In Cassie experiments, reinforcement learning followed a wider range of walking commands than the model based controller used to collect motion data. A policy also kept the robot walking after a hip motor was damaged.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.