Actuate 26 | Force Intelligence: A Path to General Dexterity by Pulkit Agrawal

In this talk

1. 0:11

   The gap between generality and performance

   Agrawal compares today’s machine learning robots with industrial systems: learned robots can handle more kinds of tasks, but often lack the speed and reliability needed for commercial use. He argues that useful products must combine performance with generality, rather than settling for one or the other.

2. 8:59

   Why manipulation depends on force

   Agrawal describes robotics as a feedback loop involving plans, control, actions, and the physics of what happens next. Vision and language models can help with planning, but physical tasks also require force feedback: pushing a chair too quickly can send it flying, and reaching for a mug too fast can cause a spill. He presents force as the key signal for controlling physical interactions.

3. 13:59

   Scaling through simulation and reinforcement learning

   Aether’s proposed vision-force-action model aims to learn physical interactions alongside visual information. Agrawal says human demonstrations are costly to scale, often slow, and tend to show successful attempts rather than failures. He proposes large-scale reinforcement learning in simulation, and says the approach has transferred to untrained objects, including raspberries.

4. 18:20

   Performance, safety, and the product vision

   Agrawal says the systems handle messy manufacturing settings and tasks such as inserting screws, including fragile glass ones. He names generality, performance, and safety around people as requirements for practical robotics. He describes building a vision-for-action model for different robot platforms alongside Aether’s own hands, and says the goal is superhuman capability.