Ep#46: ManiFlow: A General Robot Manipulation Policy via Consistency Flow Training
Gian, a PhD student at the University of Washington working with Professor Dieter Fox, discusses ManiFlow with the RoboPapers hosts; the project brings together researchers at the University of Washington and UC San Diego.
In this talk
- 0:39
Why robot policies struggle to generalize
Gian points to drops in performance when familiar simulation tasks change, and describes π0 failing to pour into an unfamiliar glass teapot or handle a new background. He says collecting enough demonstrations for every object and setting is impractical, while common robot datasets cover a narrow range of environments.
- 4:00
ManiFlow’s representations and architecture
Gian says ManiFlow achieved useful results with 30 to 50 demonstrations per task in some experiments, compared with 200 to 500 for similarly difficult ABM tasks. The policy can use image or point-cloud inputs; the real-world humanoid tests mainly use egocentric 3D from a stereo camera. Its DiT-X architecture uses cross-attention for image and language tokens, while scalar signals such as diffusion time condition the layers through adaptive normalization.
- 16:01
Consistency training for faster action generation
Gian explains that flow matching learns a vector field that can follow curved paths through action space, making inference slower. ManiFlow combines the usual velocity prediction objective with a consistency objective that encourages intermediate points to lead toward the same target. In training, 75% of the examples use the velocity loss and 25% the consistency loss; in reported tests, two inference steps performed best, with inference up to ten times faster than the compared diffusion policy.
- 31:52
Evaluation, limits, and next steps
Across 64 simulation tests in four benchmarks, Gian reports gains over flow-matching and diffusion baselines, including language-conditioned tasks in Meta-World. On RoboTwin 2.0, ManiFlow beat π0 on three of four tasks with 50 demonstrations; π0 improved as data increased, reaching 94% on list-parts with 500 demonstrations. Real-robot tests covered three setups, and Gian identifies language following and generalization to substantially different scenes as ongoing challenges that more diverse demonstrations may help address.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.