Joanne Truong on Sim2Robot: Training Robots for the Real-World with Imperfect Simulators|AIR Seminar
Joanne Truong, a Georgia Tech PhD researcher, presents her Sim2Robot research at the AI Robotics Seminar at the University of Toronto, explaining how to train robots in imperfect simulators and transfer their skills to real environments.
In this talk
- 3:04
Why simulators need careful evaluation
Truong describes simulation as a safer, faster, cheaper way to gather robot experience, while warning that agents can exploit flaws in simulated physics. To measure whether simulation results predict real robot results, her team ran 810 paired navigation experiments in a scanned Georgia Tech lab. They found that collision sliding inflated simulated performance, and disabling it improved the correlation between simulation and reality.
- 17:21
Adapt the simulator with real-world data
Truong presents bidirectional domain adaptation, which uses a small set of real-world examples to adjust both simulated robot motion and the appearance of camera images. In simulated transfer experiments, the method performed comparably to direct policy fine-tuning while using 200 times fewer samples. Most of the gains came from the first 1,000 examples, and the approach reached comparable performance 117 times faster.
- 26:03
Faster, simpler physics can transfer better
For navigation, Truong compares policies trained with simplified kinematic simulation against policies trained with detailed rigid-body dynamics. In real-world trials, the kinematic policies succeeded every time, while dynamic policies succeeded in 40% to 67% of trials and collided more often. Kinematic simulation ran faster, letting the team collect roughly ten times more experience and improving transfer across robots and simulators.
- 30:40
Use rough maps for outdoor navigation
Truong's Indoor Sim to Outdoor Real project trains a Spot robot in simulated indoor scenes, then sends it outdoors without real-world training. A policy given a hand-drawn, outdated map reached distant goals in all reported trials without collisions or human help, adapting to obstacles missing from the map. In simulation, it still succeeded 81.4% of the time with 50% map noise, and at 100% noise its performance fell back to the map-free policy's level.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.