RI Seminar: Max Simchowitz: Generative Control, Action Chunking, and Moravec’s Paradox
Max Simchowitz, a Carnegie Mellon professor who studies machine learning theory and interactive, sequential problems, explains generative control and action chunking at a Carnegie Mellon Robotics Institute seminar.
In this talk
- 1:46
Why robot imitation can fail
Simchowitz frames the challenge through Moravec’s paradox: robots have historically struggled more with physical tasks than AI systems with symbolic ones. He argues that continuous control can make imitation errors compound exponentially with task horizon, even when the dynamics and expert policy are smooth and stable. A small action error can seed instability because the learner’s policy may not preserve the expert system’s stability.
- 19:14
Action chunking can stabilize learned behavior
Action chunking predicts a sequence of actions from recent observations, rather than choosing each action independently. Simchowitz says that, in open-loop stable systems, chunks longer than a system-dependent threshold can prevent compounding error; experiments suggested that the rollout chunk length matters, with four or eight steps often used in practice. Position-tracking controllers can help by making the robot’s dynamics more stable, though chunking can reduce responsiveness.
- 27:08
Generative policies help beyond multimodal actions
Simchowitz examines whether flow-based generative policies work mainly because they model multiple possible actions. In the reactive tasks studied, flow policies and regression were competitive on most tasks, and averaging generated actions cost only about 2% of performance. A minimal iterative policy that adds noise and then corrects its predicted action performed competitively on tasks where flow models beat regression, suggesting stochasticity and iterative correction contribute to their benefit.
- 39:18
Control-shaped errors and what remains hard
Simchowitz connects generative policies to exploratory data collection, where noisy actions reveal how an expert should respond from perturbed states. His experiments suggest flow policies make more of their prediction error along a local action manifold, where errors are less damaging, and less error off that manifold. He concludes that long-horizon adaptation remains difficult, and that online reinforcement learning could help by exploring unstable directions and learning corrections.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.