2024 Fall Robotics Colloquium: Yashraj Narang (NVIDIA)

In this talk

1. 1:22

   Why assembly is hard to automate

   Narang focuses on mating and joining tasks such as peg insertion, connector insertion, and fastening. Fixed automation can be precise and reliable, but setup costs and sensitivity to variation make it difficult for smaller manufacturers to adapt. He argues that simulation-trained reinforcement learning could help address those limits.

2. 6:55

   Building a fast contact simulator

   NVIDIA’s Factory work used signed distance fields to represent detailed part geometry and speed up contact calculations. Contact reduction cut a nut-and-bolt simulation from about 16,000 contacts to 300, enabling two orders of magnitude more parallel simulations. The team used this environment to train policies and checked that simulated forces and robot torques fell within plausible real-world ranges.

3. 22:43

   Transferring assembly policies to robots

   Early deployments exposed gaps in dynamics and control, so the team added methods to account for simulation error and reduce steady-state error at deployment. On a Franka Panda, the system reached 89% success across trials combining grasping, transport, and assembly. In later work on 100 diverse assemblies, real-world deployments averaged 86.5% success after demonstrations from disassembly paths were incorporated into reinforcement learning.

4. 41:06

   Adding force and tactile feedback

   Earlier policies relied on proprioception even when vision localized the parts. For nut-and-bolt assembly, policies used force estimates from joint torques and randomized controller gains, friction, and robot dynamics during training; the combined approach improved success by about 32%, reduced peak forces by 22%, and shortened cycle times by 24%. The team also simulated GelSight-style tactile inputs and demonstrated peg alignment and insertion despite perturbations and lights being switched off.

5. 47:58

   Limits and next steps

   Narang says the methods do not yet handle large rotations, reorientation, or deformation, and their success rates and cycle times fall short of industry targets. The team’s goal is a broadly useful assembly skill that combines vision, force, and tactile sensing, generalizes across many assemblies, and can be fine-tuned in a real work cell.