Integration of Natural Language Understanding, Robot's Memory and Planning in a Humanoid Robot
A HumanoidRobots presenter explains how ARMAR 3 combines language understanding, memory, planning, and human input to prepare food and handle missing objects.
In this talk
- 0:16
Replacing missing objects by visual features
When ARMAR cannot find an item, it can propose a substitute. One method compares affordances inferred from objects’ shapes with those of the missing item; for example, one container may stand in for another if both could hold a pourable ingredient.
- 2:05
Using memory and asking for help
For a salad task, ARMAR checks places where it previously encountered corn, then consults typical locations learned from text. When neither source identifies the location, it asks the person, records that the corn is in the fridge, and generates a new plan.
- 4:11
Handling objects beyond reach
If ARMAR cannot reach or grasp a needed tool, it asks the human to hand it over. It detects contact and force in its hand before closing its grip.
- 4:52
Choosing substitutes and locating them
A common-sense method uses language data to estimate which objects are associated with actions, then compares objects by those affordances. ARMAR can suggest juice instead of soda for drinking, with the person confirming the substitution. If the juice is not at the expected spot, a learned database of common object locations helps search elsewhere; when ARMAR offers an object, it waits for contact before releasing it.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.