Ep#94: Learning a Thousand Tasks in a Day
Edward Jones, robotics professor at Imperial College London, joins PhD students Camille and Pedro on RoboPapers with host Michael to discuss their paper on teaching robots many manipulation tasks from single demonstrations.
In this talk
- 4:50
Defining tasks and separating motion phases
The researchers count each interaction between the robot and a particular object instance as a task. They distinguish a micro-skill, such as opening a door with a specific motion, from a broader action like opening. Their method separates reaching a useful position near an object from the precise movement that manipulates it.
- 6:41
Comparing learning and retrieval methods
The team compares one end-to-end behavior-cloning policy with methods that handle alignment and interaction separately. Some policies learn from demonstrations, while retrieval methods find a similar prior example using language filtering and object geometry. For alignment, retrieval maps a demonstrated pose to the current scene; for interaction, it replays the demonstrated motion.
- 17:59
How data volume and diversity affect results
The experiments vary demonstrations per task and the number of tasks represented in a fixed data budget. Decomposed policies outperform the monolithic behavior-cloning baseline with few demonstrations, while fully retrieval-based MT3 performs strongly in the low-data settings. The researchers also find that behavior cloning can benefit from more examples per task, and that greater object diversity can improve results.
- 45:54
Testing a thousand tasks and examining failures
The team taught 1,000 tasks in 17 hours using 31 skills and 42 real-world objects, then evaluated the method on those tasks and 100 unseen object instances. Across 2,200 rollouts, they found that forgiving tasks such as pulling a drawer handle often worked better than precise insertions. Failures included segmentation and pose-estimation errors, mismatched retrieved examples, and open-loop replay that could not correct mistakes.
- 59:53
Choosing methods for the data regime
The researchers argue that useful inductive biases can improve learning when demonstrations are scarce, while less constrained behavior cloning may gain an advantage as data grows. They see retrieval's limited ability to combine information across examples as a key limitation. For future work, they would explore sharing or interpolating knowledge between retrieved demonstrations.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.