Ep#87: MolmoAct 2: An open foundation for robots that work in the real world
Jiafei, an AI2 and University of Washington robotics researcher and incoming NUS assistant professor, and Henchin Fang, a recent University of Washington graduate joining Stanford for a PhD, discuss MolmoAct 2 with the RoboPapers hosts.
In this talk
- 1:19
An open foundation for robotics
Jiafei describes MolmoAct 2 as a rethink of the first MolmoAct, aimed at making an open-science robotics model that people can deploy, study, and build on. The team shares model details, training methods, and datasets, extending beyond releasing weights alone.
- 3:56
Open data for real-world tasks
Fang and Jiafei describe combining DROID data with roughly 720 hours of bimanual YAM teleoperation and filtered SO-100 and SO-101 datasets. They scraped about 5,500 community datasets and retained around 1,200 with suitable licenses and quality. Their collection emphasizes longer, practical tasks such as folding clothes and scanning groceries, with trajectory descriptions generated by a vision-language model.
- 23:17
A faster action architecture
MolmoAct 2 pairs a vision-language model with a diffusion-transformer action expert to improve action precision and generation speed. The team reports that passing key and value embeddings from corresponding vision-language model layers performed better in their ablations than the alternative connections they tested. The Think variant adds depth perception tokens and updates only image regions that change, which the team says makes inference about twice as fast without a performance drop.
- 43:44
Evaluation and community use
The team reports top results among VLA models on RoboTwin and RoboCasa at the time of evaluation, and describes real-world comparisons across five policies. They assess trajectory qualities such as jerkiness, path length, and completion time alongside success rates. Jiafei also points to community deployments, including SO-100 robots, a live cup-stacking demo, and projects using the model for continuous evaluation and commercial services.
- 54:31
What comes next
The team estimates that MolmoAct 2 used less than half a million dollars of compute, attributing the lower cost than the earlier project to validating components before the main training run. Jiafei says a future system should work zero-shot in unfamiliar homes and may use an orchestrator over multiple specialist policies. The team also found that a naive 50-50 mix of simulation and real-world training performed worse than either source alone.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.