World Models: The Frontier of Physical AI | Physical AI Day at SIGGRAPH 2026

In this talk

1. 0:26

   Physical AI needs scalable data

   The presenter says AI has changed digital work, while factories, farms, vehicles and other physical systems remain a large opportunity. Teleoperation produces accurate demonstrations but is slow to scale, and simulations take work to build. World foundation models could generate robot-centered scenarios and turn computing time into training data.

2. 4:19

   Cosmos combines understanding and generation

   The presenter describes Cosmos as a foundation for developers to build world models for their own needs. Cosmos 3 is presented as supporting text, images, video, audio and action, with an autoregressive tower for reasoning and a diffusion tower for generating outputs. The models are offered in three sizes: H at 4 billion parameters for on-device use, Nano at 16 billion as a starting point, and Super at 64 billion for higher-fidelity results.

3. 26:34

   A shared action language for different robots

   Cosmos treats action as a core modality and represents movement with geometric elements such as body pose, end-effector pose and grasp state. The presenter says this lets vehicles, robot arms, humanoids and human-view videos map into a common scheme. The model can support dynamics, inverse dynamics and policy, connecting actions, observed outcomes and decisions.

4. 31:29

   From world models to real-time robot control

   The presenter frames dynamics as predicting effects, inverse dynamics as inferring actions from observed changes, and policy as selecting actions for a task. NVIDIA's Edge Policy demonstration runs on a Jetson Thor embedded device at about 130 watts, using camera input to control a robot arm in real time. Cosmos can also be adapted with a developer's own data, while distillation is described as cutting generation from 50 steps to four.

5. 35:47

   An open platform with an expanding roadmap

   NVIDIA's Cosmos GitHub repository brings model releases, inference options, fine-tuning and distillation scripts together. The presenter says the platform includes open models, data and frameworks, and previews Cosmos 3.5 with multi-view modeling and a mixture-of-experts approach. A coalition of companies is expected to contribute research, data and evaluation.