Bingjie Tang on Transferring Contact-Rich Assembly from Simulation to Reality | Toronto AIR Seminar

In this talk

1. 1:35

   Why use simulation for assembly

   Tang describes assembly as precise, contact-heavy work that often depends on costly hardware and carefully engineered fixtures, lighting, and trajectories. She explains that simulation can reduce this burden, but complex part geometry, inaccurate physics, and controller noise make reliable sim-to-real transfer difficult.

2. 10:09

   Training insertion policies in simulation

   The team split assembly into picking, placing, and inserting, and trained task-specific policies with PPO. To improve insertion learning, they filtered and down-weighted episodes with excessive mesh overlap, used signed distance field rewards to handle part symmetry, and gradually widened the range of starting positions. Combining the overlap filter and weighting reached 85% success in simulation; the SDF reward achieved 88.6% success in its reported evaluation.

3. 23:35

   Preparing policies for real robots

   With the three training techniques combined, insertion policies reached 88.6% success for hose assembly and 82% for gear assembly in the stated evaluations. Tang also introduced PLAI, which accumulates policy actions at deployment time; on a Franka robot, it reduced steady-state reaching error to about 2 mm despite friction and gravity effects.

4. 29:12

   End-to-end assembly and remaining limits

   The real-world system achieved 99% success on picking, 83% on insertion, and 89% across sequential pick, place, and insert trials. Tang noted that parts could slip because the system did not track objects after grasping, and that searching around a socket could be risky for delicate components. She said industrial use would require higher speed and accuracy, closer to 99.9% success, and support for more part types.