RoboBoom
Fellows Forum· September 29, 2026· 40:11

Keynote: Foundation Models for Robotics and the Physical World | Day 2

Ming-Yu Liu, who leads NVIDIA’s Cosmos Lab, explains how Cosmos models and NVIDIA’s computing platform can support robotics and other physical AI applications at the Fellows Forum.

In this talk

  1. 0:40

    Build flexible computing for specialized AI

    Liu describes NVIDIA’s work across GPUs, CUDA, models, and reference applications, saying these tools help developers build products while informing future hardware. He argues that different applications need specialized models and architectures, yet flexible computing makes it easier to shift resources as business needs change. A single general model may be convenient, but a collection of specialized models can be harder to operate, so teams need to balance performance with deployment complexity.

  2. 11:52

    Coordinate agents across physical industries

    Liu sees physical AI as a major opportunity across agriculture, transport, factories, healthcare, and other sectors. His factory example links planning agents with robots, mobile platforms, trucks, assembly stations, and inspection systems to fulfill an order across multiple sites. He presents Cosmos as a shared base for specialized agents, with an orchestration agent coordinating their work.

  3. 18:16

    Use Cosmos for data, simulation, and adaptation

    NVIDIA offers Cosmos models and training resources that developers can adapt for commercial use, along with datasets, code, and partner examples. Liu highlights synthetic data for rare events, such as a landslide on a car’s route, and simulated environments for evaluating many driving-policy checkpoints before using real vehicles. He says Cosmos is intended as a reusable starting point for application-specific agents.

  4. 23:04

    Bring perception, world prediction, and action together

    Liu says physical AI needs language understanding as well as a representation of how the world changes. He describes Cosmos 3 as combining a language-oriented transformer with a generative model that can produce images, video, sound, and actions, with action treated as a core part of the model. The system aims to connect future-action prediction, inference about actions from observations, and policy generation, across embodiments such as cars, robot arms, and humanoids.

  5. 25:42

    Scale models to devices and keep improving

    Liu says Cosmos 3 combines capabilities from earlier models, while planned Cosmos 3.5 work includes agent features, multi-camera generation, lighting, and stronger reasoning within a prompt. The released model sizes range from a 4-billion-parameter H model designed for embedded hardware to 16-billion Nano and 64-billion Super versions. In response to questions, he identifies data and compute as continuing constraints for robotics, and says agents and specialization will matter, though the most cost-effective role for LLMs remains unsettled.

Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.

More talks