Chelsea Finn, a Stanford professor, discusses how robots can generalize through broader training data and adapt to unfamiliar settings at a CMU Robotics Institute seminar.
Finn describes training a language-conditioned robot on demonstrations for 100 tasks spanning nine skills. It completed some held-out tasks without seeing those exact instruction and object combinations, with nonzero success on 20 of 28 tasks and a 32% average success rate. She says fine-grained control, rather than interpreting instructions, was the main bottleneck.
Finn argues that robotics research often collects data for one project and then leaves it unused, unlike fields that reuse large datasets or pretrained models. She describes an effort to gather robot data across 50 or more real households, while acknowledging that differences in robots, cameras, and setups make shared data difficult. She invites the community to help shape the effort.
Finn shows how reinforcement learning fine-tuning can improve grasping in conditions that differ from training, such as a gripper shifted by 10 centimeters. In that example, success rose from 43% to 98% after adaptation; across other scenarios, 25 grasps helped in four of six cases, and 800 grasps brought results closer to the original baseline. For longer tasks, she describes a method that uses prior experience to help an agent recover from unfamiliar states during a single episode.
Finn reports that wrist-mounted cameras improved manipulation performance under changes such as unfamiliar distractors and table heights. In one real-robot evaluation, success was around 52% with a hand-centric camera, compared with 15% to 20% for a third-person view. She says her lab now uses wrist cameras for manipulation projects.
Finn argues that broad generalization should not come at the cost of task complexity. She presents a bimanual teleoperation setup under $20,000 that can unwrap candy, zip a hoodie, and package items in a Ziploc bag. The setup uses off-the-shelf and 3D-printed parts and was planned for open source release.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
Chelsea Finn of Physical Intelligence discusses building reliable, general-purpose robots, drawing on office coffee service, factory box assembly, and laundry-folding examples.
Chelsea Finn, cofounder of Physical Intelligence, presents at a Y Combinator event on building reliable, general-purpose robots for real-world work.
Chelsea Finn, co-founder of Physical Intelligence, discusses building general-purpose robots at a Y Combinator event.
Chelsea Finn, co-founder of Physical Intelligence and Stanford associate professor, discusses building general-purpose robots with the No Priors hosts.
Stanford AI and robotics professor Chelsea Finn talks with host Peter on The Robot Brains Podcast about helping AI systems adapt to changing conditions, with examples from language models and robots.
Stanford University and Google Brain Professor Chelsea Finn presents her RI Seminar at Carnegie Mellon Robotics Institute on scaling data and algorithms for robot learning, introduced by host Wen.
