Panel: Robotics & World Models
Guru Chahal moderates Berkeley RDI panelists Anastasis Germanidis, Jim Fan, Sergey Levine, Michael Spranger, and Wei Zhan on world models, robotics data, simulation, and commercial deployment.
Sergey Levine, co-founder and researcher at Physical Intelligence, discusses robotic foundation models, learning from experience, and the challenges of building useful general-purpose robots with Invest Like The Best host Patrick O’Shaughnessy.
Levine says Physical Intelligence aims to build models that control different embodied systems across many tasks. The case for generality is that learning across tasks and robots could build shared physical understanding, much as language models draw on broad data. That foundation could let people create new robotic applications without building the full intelligence stack themselves.
Levine describes robotic learning as a decades-old idea, with early end-to-end driving systems in the 1980s and deep reinforcement learning in the 2010s as milestones. Physical Intelligence adapts language models into vision-language-action models using web, image, and robot data. The team uses intermediate reasoning to apply prior knowledge to unusual scenes, then reinforcement learning to improve practiced skills, as in its espresso-making demo.
Levine says the team’s robot uses three cameras and no touch or force sensors, and that learning can compensate for limited sensing. He expects useful robots to gather more data as they work, though the right data source may vary by domain. In one experiment, adding high-level language labels to a robot’s experience improved generalization without adding more teleoperation actions, suggesting people may be able to coach robots through instructions.
Levine sees unpredictable home environments as a major technical challenge: robots must respond sensibly even when they cannot complete a task. He also expects care for children and older adults to be difficult because these jobs combine physical skill with sensitive human interaction. In workplaces, he anticipates robots working alongside people in ways similar to coding tools that increase software engineers’ productivity.
Levine argues that accessible models could let many people prototype robotic applications, and says Physical Intelligence has open-sourced models and wants other companies to experiment with them. He is still uncertain how much deployment will rely on demonstrations, autonomous experience, or a combination, and says those choices affect how businesses should prepare. The team is currently studying how robots should represent and reason about scenes and tasks between high-level instructions and low-level actions.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
Guru Chahal moderates Berkeley RDI panelists Anastasis Germanidis, Jim Fan, Sergey Levine, Michael Spranger, and Wei Zhan on world models, robotics data, simulation, and commercial deployment.
Sergey Levine, UC Berkeley EECS professor and Physical Intelligence co-founder, discusses robot foundation models at an EMBER Center seminar at UC Berkeley.
Professor Sergey Levine of UC Berkeley and Physical Intelligence speaks with the AI Agent Frontier Seminar about robotic foundation models and how robots can learn to act reliably in the physical world.
Sergey Levine, a robotics researcher at UC Berkeley and co-founder of Physical Intelligence, explains how robot foundation models use diverse cross-robot data, language and vision backbones, and continuous action models to handle increasingly complex tasks.
Sergey Levine, co-founder of Physical Intelligence and UC Berkeley professor, discusses the path to general-purpose robots with Dwarkesh Patel.
Sergey Levine, robotics researcher and co-founder of Physical Intelligence, discusses generalist robot foundation models and their training at a University of Toronto Robotics Institute seminar.