Continuous Learning Platform for Physical AI | Steve Xie (Lightwheel)

In this talk

1. 3:52

   Why robotics needs continuous learning

   Xie compares how children learn from demonstrations, trial and error, and feedback with how robots should learn. He argues robotics lacks a large-scale loop for gathering data and feedback from real work, and estimates it may need far more scale than autonomous driving because robots interact physically with many objects.

2. 12:10

   Two data pyramids and a learning loop

   Xie proposes separate data pyramids for training and evaluation. For training, he suggests roughly 99.9% of data come from human experience and simulation, with about 0.1% from real robot collection. For evaluation, he says most trials should happen in simulation, where robots can fail safely before deployment.

3. 16:05

   Building real-to-simulation infrastructure

   Xie describes SimFoundry as Lightwheel’s infrastructure for turning real-world experience into simulated scenes and tasks. Its components include accurate physics, a factory for measuring real-world physics, and tools for generating assets and tasks at scale. He reports that a simulated robot pulling a microwave door matched real results by more than 99% on the first try.

4. 21:09

   Data, evaluation, and deployment products

   EgoSuite Open 100K is described as 100,000 hours of multimodal, annotated egocentric data, including hand and full-body poses and language annotations. RoboFinals turns human experience into simulation benchmarks, while RoboStack deploys robots in settings such as hospitals and gathers feedback on errors. Xie describes using that feedback to collect more data, refine models, test again in simulation, and redeploy.

5. 27:24

   An open platform for the ecosystem

   Xie says robotics is too difficult for any one company to solve alone, so Lightwheel aims to build an open continuous learning platform. He invites collaboration across simulators, hardware companies, model developers, researchers, and sensor makers, including through shared software, data, evaluations, and standards.