How Physical AI Learns Across Language, Video and Action, Ming-Yu Liu
Ming-Yu Liu, who leads research at NVIDIA, discusses how Cosmos 3 combines language, video, audio, and action for physical AI with Tim on Machine Learning Street Talk.
Ming-Yu Liu, who leads NVIDIA’s Cosmos Lab, explains how Cosmos models and NVIDIA’s computing platform can support robotics and other physical AI applications at the Fellows Forum.
Liu describes NVIDIA’s work across GPUs, CUDA, models, and reference applications, saying these tools help developers build products while informing future hardware. He argues that different applications need specialized models and architectures, yet flexible computing makes it easier to shift resources as business needs change. A single general model may be convenient, but a collection of specialized models can be harder to operate, so teams need to balance performance with deployment complexity.
Liu sees physical AI as a major opportunity across agriculture, transport, factories, healthcare, and other sectors. His factory example links planning agents with robots, mobile platforms, trucks, assembly stations, and inspection systems to fulfill an order across multiple sites. He presents Cosmos as a shared base for specialized agents, with an orchestration agent coordinating their work.
NVIDIA offers Cosmos models and training resources that developers can adapt for commercial use, along with datasets, code, and partner examples. Liu highlights synthetic data for rare events, such as a landslide on a car’s route, and simulated environments for evaluating many driving-policy checkpoints before using real vehicles. He says Cosmos is intended as a reusable starting point for application-specific agents.
Liu says physical AI needs language understanding as well as a representation of how the world changes. He describes Cosmos 3 as combining a language-oriented transformer with a generative model that can produce images, video, sound, and actions, with action treated as a core part of the model. The system aims to connect future-action prediction, inference about actions from observations, and policy generation, across embodiments such as cars, robot arms, and humanoids.
Liu says Cosmos 3 combines capabilities from earlier models, while planned Cosmos 3.5 work includes agent features, multi-camera generation, lighting, and stronger reasoning within a prompt. The released model sizes range from a 4-billion-parameter H model designed for embedded hardware to 16-billion Nano and 64-billion Super versions. In response to questions, he identifies data and compute as continuing constraints for robotics, and says agents and specialization will matter, though the most cost-effective role for LLMs remains unsettled.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
Ming-Yu Liu, who leads research at NVIDIA, discusses how Cosmos 3 combines language, video, audio, and action for physical AI with Tim on Machine Learning Street Talk.
NVIDIA founder and CEO Jensen Huang presents the company’s AI strategy and product announcements at GTC Berlin, with European partners and customers.
NVIDIA CEO Jensen Huang discusses Europe’s AI economy, NVIDIA’s AI infrastructure and agent platforms, and physical AI at GTC Berlin, with examples from European industry and NVIDIA partners.
The host and main speaker from Robotics Central explains Kawasaki’s staged plan to deploy Kaleido humanoid robots, and how Japan’s AI funding, Nvidia computing, and human remote operators fit into that effort.
Amulia Vishban, founder of Techable Ventures, discusses investing in physical AI with Dominic Madori Davis on TechCrunch’s Equity podcast.
Les Carpus, global head of physical AI for NVIDIA’s Inception program, discusses robotics startups and the path to a ChatGPT moment with TechCrunch reporter Tim at Disrupt 2026.