Robotics Colloquium: NVIDIA Seattle Robotics Lab
Lars, Jan, and Yushuan of NVIDIA’s Seattle Robotics Lab discuss industrial assembly, robot foundation models, and interactive world models at the Paul G. Allen School robotics colloquium in Seattle.
In this talk
- 6:26
A mixed toolkit for precision assembly
The assembly team is working on inserting cables and a bus bar into NVIDIA GB300 systems, where a damaged heat sink can ruin a $100,000 unit. They combine vision, mechanical design, motion planning, and reinforcement learning, choosing methods to fit each task. For bus-bar insertion, motion primitives with local adjustments are faster than reinforcement learning, which still falls short on cycle time.
- 11:22
Teaching robots and planning around collisions
Kinesthetic teaching lets people guide a robot by hand while a controller keeps demonstrations within the speed and acceleration limits a learned policy can reproduce. The approach matches or exceeds a baseline while reducing human interventions and cycle time. The Q-Arm planner computes multi-arm collision-free paths in about two seconds for a welding example, compared with two days for the cited alternative.
- 25:30
Evaluating robot foundation models
Jun compares vision-language-action models, which build on vision-language models, with world-action models that use video-generation backbones. He argues that high simulation benchmark scores can reflect shortcuts rather than real-world competence, and presents RoboLab as a robot-agnostic evaluation platform with about 120 pick-and-place tasks. In one example, a Cosmos 3 Nano policy improved from 4.4% to 39% on RoboLab and then handled tasks beyond the benchmark on a real robot.
- 33:30
Reasoning, speed, and model deployment
Jun reports that general-purpose model Astra outperformed the leading robot-focused model in the lab’s benchmark, while its slow reasoning pauses make real-time operation impractical. The ARC training recipe adds detailed causal annotations and counterfactual losses to help robot policies follow instructions instead of relying on visual shortcuts. Using the same robot action data, the team reports substantial benchmark gains, including roughly 30% on vague instructions.
- 45:22
Interactive world models for training and evaluation
Yushuan presents an interactive world simulator that predicts robot interactions at about 15 frames per second, including subtle cases such as a failed grasp or a rope missing a clip. The team trained it on six hours of teleoperated play data and reports that simulated evaluation correlates with real-robot performance. They also use the model to generate training data, with mixed simulated and real data producing comparable results.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.