How do you train autonomous robots? Toyota starts in factories.
A Toyota Research Institute robotics team presenter describes how Toyota aims to train adaptable robots in factories alongside team members.
David Watkins, a robotics researcher whose talk covers work at RAI and Toyota, discusses demonstration capture, robot hardware and data collection at Toyota’s 100-robot facility.
Watkins describes using inexpensive UMI-style interfaces to collect demonstrations that support relative-position control. At RAI, his team redesigned the handheld mechanism and trigger for smoother, easier use, and built a UMI-style gripper for Spot. Comparing their data with Stanford’s Cup in the Wild dataset, they found Stanford’s demonstrations included more varied paths, shorter episodes and faster movements.
With 1,024 demonstrations, the team trained a policy that handled cups in unseen locations, a different cup color and different surfaces. For a pouring task, Watkins says the policy’s memoryless behavior made the bottle’s transparency important: it could detect when pouring was done with a clear bottle, but not an opaque one. He describes a training approach that combines broad pretraining, curated demonstrations and a small amount of task-specific fine-tuning.
The team trained a base policy from demonstrations collected with YuMi, evaluated where it struggled, then added teleoperated examples for those gaps. They trained a router to distinguish the data sources and modified the diffusion policy to better combine state and visual input. Watkins also describes Koala grippers, which support different grasping modes for tasks such as handling tools, separating small objects and picking up paper.
At Toyota, Watkins describes Data Factory One, a facility in Watertown, Massachusetts, with 100 robots operated by teleoperators in different regions. The robots provide RGB, proprioceptive data and four point clouds from head, torso and wrist cameras; teleoperators see those views alongside the robot’s live state. The team trained a clothing-folding policy exclusively on this data and worked to keep the fleet online and provide researchers with curated, labeled data.
Watkins argues that robot hardware and software should be developed together, and that teams should carefully curate data and diversify its sources. In the Q&A, he says teleoperators improve with experience, while force data can help with dexterous contact tasks such as peg insertion. He notes that visual input alone may not capture contact forces, and mentions audio and tactile sensing as possible ways to clarify the robot’s interaction with objects.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
A Toyota Research Institute robotics team presenter describes how Toyota aims to train adaptable robots in factories alongside team members.
James Kuffner of Toyota Research Institute reflects on 30 years in robotics and the challenges of building safe, intelligent machines, speaking at an IEEE Robotics and Automation Society event.
Dr. Gill Pratt, Chief Scientist at Toyota Motor Corporation and CEO of Toyota Research Institute, discusses technology strategy and carbon reduction at a leadership talk hosted by Autos2050.
Akio Toyoda, president of Toyota Motor Corporation, and Gill Pratt, incoming CEO of the Toyota Research Institute, discuss Toyota’s plans for AI, safer and more accessible mobility, and robotics at a press conference with each other.
Akio Toyoda, president of Toyota Motor Corporation, and Gill Pratt, CEO of the new Toyota Research Institute, outline plans for AI research at a Toyota press conference in Japan.