Ep#78: Three Eras of Robot Learning

In this talk

1. 2:07

   Existence proofs through robot experience

   Tetsuya traces the first era to deep reinforcement learning breakthroughs such as DQN and AlphaGo, which inspired attempts to learn directly from real robots. QT-Opt used the cross-entropy method to maximize Q-values for continuous robot actions, while concurrent control and simulation-to-real tools helped run large robot fleets. Later work expanded into multitask learning and play data, but scaling expert demonstrations became a turning point.

2. 9:38

   Why expert data changed the recipe

   The team paused publication for about a year and a half to collect demonstrations across hundreds of tasks using nearly 10 robots and 10 operators. Reinforcement learning improved with experience but was difficult to maintain across distributed systems, while behavior cloning often plateaued around 60 to 80 percent. After infrastructure improvements, imitation learning reached 90 to 95 percent, giving the team confidence to scale data collection.

3. 25:48

   Foundation models enter robotics

   Tetsuya describes the convergence of large robot datasets with pretrained language and vision-language models. RT-1 trained a 5-million-parameter transformer on about 87,000 trajectories, while SayCan combined language-model planning with robot affordances. Follow-on work used VLMs for data relabeling, developed vision-language-action models, and pooled data across 34 institutions through Open X-Embodiment.

4. 41:31

   Scaling models, data, and embodiments

   In the current scaling era, Tetsuya points to larger backbones, more data, and more capable platforms such as ALOHA and humanoids. Gemini Robotics added embodied reasoning for spatial tasks, and Gemini Robotics 1.5 explored text-based planning and motion transfer across different robot embodiments. He also highlights scaling post-training, evaluation, egocentric human data, and real-world product deployments.

5. 53:08

   Open questions about research and adoption

   Tetsuya sees academia contributing rigor, analysis, and foundational ideas, while industry increasingly bears the cost of proving capabilities at scale. He says limited data sharing can slow progress, though he understands why companies protect work built through substantial investment. He expects a robotics consumer moment to require both technical advances and a useful, accessible product, and sees video action models and human data as promising research directions.