Dieter Fox, senior robotics research director at NVIDIA and professor at the University of Washington, reflects on robotics’ progress and the prospects for open-world robots, with the IEEE Robotics and Automation Society.
Fox considers why robotics has not had a ChatGPT-like leap. Looking back to his start in 1994, he describes early sonar-based mapping and navigation as fragile, then explains how 2D LiDAR enabled substantial gains, including museum tour guides. Even decades later, indoor navigation still struggles with changing surroundings and semantic understanding.
Probabilistic methods improved localization, SLAM, and planning steadily, but Fox says sensor and motion models alone could not solve robotics’ challenges with visual and semantic understanding. Later, Velodyne 3D scanners helped drive autonomous vehicle development, while depth cameras advanced object modeling and manipulation. Expectations that self-driving and indoor manipulation would be solved quickly proved premature.
Deep learning made it possible to learn visual features from data, but real-world reinforcement learning remains difficult, so behavior cloning is widely used. Fox sees generative AI, including CLIP-style visual representations and progress in image and video understanding, as a new source of open-world reasoning capabilities. He expects major progress, though not a fully capable general-purpose robot in the next two or three years.
Fox urges robotics researchers to help develop the models and use simulation to produce training data. In response to questions, he says robots must recover from errors by trying again or asking people for help, and points to possible uses in homes, hospitals, and industry. Safety work must address both physical hazards and harmful consequences of seemingly ordinary actions, such as pressing a button.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
Dieter Fox, professor at the University of Washington and robotics research lead at NVIDIA, discusses where robotics stands relative to generative AI and how simulation could help train more capable robots, during a CMU Robotics Institute seminar.
Caelan Garrett, Ankit Goyal, and Iretiayo Akinola, research scientists at NVIDIA Research, present robotics work at the University of Washington Robotics Colloquium hosted by Dieter Fox.
Dieter Fox, senior research director at NVIDIA and professor at the University of Washington, discusses building reusable robot manipulation skills in a talk hosted by MIT Robotics.
Dieter Fox, a University of Washington professor who also leads robotics research at NVIDIA, presents joint work from those teams at GRASP Lab on robot manipulation, with host Dan.
University of Washington professor and NVIDIA Senior Director of Robotics Research Dieter Fox presents his lab’s work on robot manipulation at the University of Toronto Robotics Institute seminar.
