Mac Schwager, a robotics researcher speaking for his lab at MIT Robotics, examines whether vision-language-action policies generalize and what might make them more reliable.
Schwager acknowledges impressive demonstrations such as Physical Intelligence’s robot folding laundry, then questions how well those results generalize. He says his lab’s studies find that policies are brittle and often memorize training examples, especially after fine-tuning.
In benchmark examples, a policy trained to put a book in a caddy struggles when asked to put a mug there, and it also makes errors when objects switch sides. Schwager calls these concept and spatial lock-in. In his lab’s experiments, L2 regularization of the vision encoder and contrastive prompt guidance improved results, with some conditions reaching 19 successes out of 20.
Schwager reports that diffusion policies’ inferred action sequences closely match chunks from their training data. He argues that this can support stable behavior: closed-loop feedback lets the robot switch among familiar chunks as new images arrive. In tests with visual disturbances, diffusion policies stayed more consistent than ACT, while ACT sometimes produced erratic actions.
Using sparse autoencoders to inspect model features, Schwager’s group found high rates of memorization: 97 to 98 percent for LIBERO, 95 or 89 percent for DROID depending on the probe, and 99.5 percent for OpenVLA. He describes experiments that expanded drone training with 200 synthetic gate demonstrations, raising gate success from 70 to 85 percent. A task progress model also helped select cleaner T-shirt-folding data for training.
Schwager urges researchers to treat impressive results as provisional until they can be reproduced with shared code, data, and tools. He says current policies lack memory and metacognition, and argues that robots need ways to assess their progress and learn from real-world experience. He frames generalist robotics as an open research problem.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
Haruki Nishimura, a Stanford PhD researcher in the Multi-Robot Systems Lab, presents his thesis on planning robot trajectories under uncertainty in interactive environments, with advisor Mac Schwager and the defense committee at Stanford.
