CoRL 2024 Industry Panel

In this talk

1. 1:01

   Four companies, four research perspectives

   Nicholas Heess described Google DeepMind’s work on dexterity and robotic foundation models, with questions around data, generalization, autonomous learning, and simulation. Russ Tedrake outlined Toyota Research Institute’s shift toward large behavior models and its Boston Dynamics partnership. NVIDIA’s Yashash presented its compute and software platforms, including Isaac Lab and GR00T, while Physical Intelligence’s Sage described its π₀ vision-language-action model trained on data from multiple robots.

2. 11:41

   What is driving optimism

   Panelists pointed to more capable robot movements, early signs of behavioral generalization, and ways to bring world knowledge from foundation models into robotics. They also cited lower-cost hardware, improved simulation, real-to-sim methods, and video or latent dynamics models. The optimism came with a caveat: these pieces are still incomplete and need more work together.

3. 17:40

   Building trust while pursuing long-term research

   Panelists said research takes time and capital, while public trust can suffer when demos are hard to verify. They called for reproducible demonstrations and clearer reporting of conditions, success rates, and evaluation results. Nicholas Heess noted that evaluating many tasks is costly, making better evaluation methods a community need.

4. 25:41

   Where robots may find early uses

   Physical Intelligence said its first π₀ release used real-world data and evaluations, including hundreds of hours of robot operation, and described simulation as useful for faster feedback but challenging for diverse dexterous tasks. NVIDIA’s Yashash said both homes and industry pose difficult problems: homes are unstructured, while industrial settings demand precision, reliability, and fast cycle times. Panelists identified logistics and truck unloading as areas where learning could yield near-term returns, while cautioning against narrowing general-purpose research too early.

5. 35:10

   Lessons from autonomous driving and the road ahead

   Panelists called autonomous driving a technological success and highlighted lessons in real-world data collection, curation, labeling, and evaluation, while noting commercial success remains to be seen. They contrasted driving’s rare-event safety challenges with the variety of tasks and requirements facing general-purpose robots. The panel ended on a broadly optimistic note, alongside concern that hype could trigger backlash and undermine real progress.