Teaching a robot its limits for open-ended chores
An MIT CSAIL researcher discusses PROGRESS, a system that combines language models with constraint checking to help robots complete open-ended manipulation tasks safely.
In this talk
- 0:14
Account for robot constraints
Language and vision-language models can suggest actions, but robots must also obey limits involving movement, reach, and collisions. The research goal is to make decisions with these models while respecting those physical constraints.
- 0:46
Search procedural plans
PROGRESS asks a language model to produce code describing a family of ways to achieve a goal. A constraint solver searches that family, testing action sequences in simulation and checking whether they satisfy the constraints.
- 1:17
Test unfamiliar manipulation tasks
The experiments examined whether a real robot could handle drawing and object arranging without prior exposure to those tasks. Examples included drawing shapes such as stars and arranging fruit, blocks, and bowls into lines.
- 1:48
Improve success by checking constraints
Compared with PROGRESS, baseline methods often collided with objects, missed the goal, or could not reach items. The researchers report higher success with explicit constraint reasoning, including a task to put each block into a bowl of the same color.
- 2:14
Extend toward household assistance
The approach aims to let robots work safely toward human-specified goals in varied environments. Future directions include settings where the robot has incomplete information or can move, with household tasks such as cooking and cleaning as longer-term applications.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.