Panel: Robotics & World Models
Guru Chahal moderates Berkeley RDI panelists Anastasis Germanidis, Jim Fan, Sergey Levine, Michael Spranger, and Wei Zhan on world models, robotics data, simulation, and commercial deployment.
Professor Sergey Levine of UC Berkeley and Physical Intelligence speaks with the AI Agent Frontier Seminar about robotic foundation models and how robots can learn to act reliably in the physical world.
Levine contrasts a language model that can explain how to do household chores with a robot that can actually perform them. Everyday tasks such as folding laundry and cooking require planning across semantic, visual, spatial, and motor levels. He argues that learning through real-world interaction can teach AI about physical phenomena beyond human-made environments such as the web.
Planning through code can handle logical tasks, but many hand movements rely on skills people cannot easily describe step by step. First-generation vision-language-action models such as RT-2 represented actions as text, while newer designs such as π₀ use a dedicated action expert to produce continuous movements. Levine shows π₀ folding laundry and recovering from some mistakes.
Embodied chain of thought lets a vision-language-action model identify objects and break instructions into subtasks. Levine says adding this reasoning improved OpenVLA's performance, even compared with Google's much larger RT-2-X. In π₀.₅, people can teach tasks through verbal instructions, and the model can use those lessons to act autonomously in unfamiliar homes.
The RT-X project pooled robot data from more than 30 research labs; its generalist model averaged 50% better performance than the labs' task-specific models in their own settings. Levine describes a training recipe that combines broad, diverse pretraining with smaller, high-quality post-training data. Pretraining also helps robots recover from errors, while data from other robot types can benefit platforms that contributed little data themselves.
Levine presents reinforcement learning as a way to improve the robustness and throughput of generalist robot policies. Physical Intelligence's π-Star 0.6 uses a value function to judge whether actions move a task toward success, then uses new autonomous experience to refine that judgment. Examples include making espresso for about 13 hours and assembling chocolate boxes, with throughput roughly doubling on several tasks compared with supervised fine-tuning.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
Guru Chahal moderates Berkeley RDI panelists Anastasis Germanidis, Jim Fan, Sergey Levine, Michael Spranger, and Wei Zhan on world models, robotics data, simulation, and commercial deployment.
Sergey Levine, UC Berkeley EECS professor and Physical Intelligence co-founder, discusses robot foundation models at an EMBER Center seminar at UC Berkeley.
Sergey Levine, co-founder and researcher at Physical Intelligence, discusses robotic foundation models, learning from experience, and the challenges of building useful general-purpose robots with Invest Like The Best host Patrick O’Shaughnessy.
Sergey Levine, a robotics researcher at UC Berkeley and co-founder of Physical Intelligence, explains how robot foundation models use diverse cross-robot data, language and vision backbones, and continuous action models to handle increasingly complex tasks.
Sergey Levine, co-founder of Physical Intelligence and UC Berkeley professor, discusses the path to general-purpose robots with Dwarkesh Patel.
Sergey Levine, robotics researcher and co-founder of Physical Intelligence, discusses generalist robot foundation models and their training at a University of Toronto Robotics Institute seminar.