All Data. Any Quality. Fully Utilized by one Latent World Action Model

In this talk

1. 0:02

   Scaling embodied learning with varied data

   The presentation introduces LDA as a way to train robots using knowledge from open-world physical settings. It is intended for tasks such as retail picking, household laundry, and factory lifting. A cooking demo shows a robot responding to changing instructions.

2. 1:24

   Matching data to learning tasks

   The presentation argues that reliance on expert demonstrations limits existing vision-language-action and world models. LDA combines policy learning, forward and inverse dynamics, and video forecasting in a multimodal diffusion transformer. Expert demonstrations support all four tasks, lower-quality behavior data supports dynamics learning, and human video without action labels supports video forecasting.

3. 2:36

   Learning dynamics in latent space

   LDA represents dynamics in a compact latent space to focus on structure rather than visual details such as lighting, texture, and color. The presentation reports stronger scaling than pixel-based models on contact-rich, dexterous, and long-horizon manipulation.

4. 2:43

   Reported comparisons and scaling direction

   The presentation says LDA outperforms π₀.₅ and Groot by up to 48%. It compares LDA's approach to GPT's use of text from varied sources and describes continued model scaling as the next step.