How Open World Models are Powering the Next Breakthroughs in Physical AI | NVIDIA GTC San Jose 2026

In this talk

1. 0:12

   Solving the physical AI data gap

   The presenter frames physical AI as the next stage after generative and agent AI, and says robots lack the abundant training data available to internet models. Cosmos can learn from internet-scale data and help generate robot-focused examples, alongside physics engine data and real robot observations paired with actions.

2. 5:01

   Training robot policies in simulated worlds

   The goal is a training environment where an agent practices tasks across varied settings before deployment. For example, a policy could learn to cut a cucumber in many kitchens; varying environments and tasks can help it generalize to unfamiliar situations. The system needs both a way to simulate what happens next and a way to judge whether the task succeeded.

3. 10:28

   Cosmos models and robot policy results

   The presenter describes Cosmos Reason for video understanding, Predict for forecasting future video, and Transfer for turning control signals into realistic scenes. A Cosmos Predict based policy achieved a reported 98.5% average success rate on Libero and performed strongly on RoboCasa. In real robot examples, it handled tasks such as folding a T-shirt with 15 demonstrations; model-based planning improved completion scores by an average of 12.5 points in challenging settings.

4. 23:58

   From world models to closed-loop simulation

   Cosmos Transfer can use depth, edges, segmentation, and other controls to generate realistic video, including multiple camera views. NVIDIA's AlphaDream driving simulator connects steering actions to changing visual scenes, and can run with the Alpamayo 1 driving policy. The presenter also previews Cosmos 3, which combines reasoning and generation and can take and produce inputs such as video, sound, text, and actions.

5. 29:50

   Applications and the compute-for-data approach

   The presenter describes applications including surgical video generation, factory camera monitoring, and a physical data factory that combines physics simulation, generated variations, and video annotations. The central approach is to spend compute to create data that is difficult to collect in the physical world. Cosmos is intended to support physical agents by both interpreting their surroundings and modeling possible futures.