Physics-based Manipulation With and Around People: Siddhartha Srinivasa (Allen School)

In this talk

1. 2:01

   Building robots that handle unfamiliar objects

   Srinivasa describes robotic manipulation as turning everyday human skills into algorithms for robots. His team trained a robot to identify Oreos, separate the cookie halves, and scrape off the filling, succeeding about three times in ten attempts. He says the broader aim is to build robots that can adapt to objects and settings they were not designed for.

2. 7:09

   Treating people as collaborators

   When robots treated nearby people as obstacles, people felt ignored and sometimes responded in ways the robots could not handle. Srinivasa argues that robots should reason about people as agents with intentions and goals. He uses cooking together as an example of why robots need to work alongside people without requiring them to leave the space.

3. 11:34

   Modeling interaction and influence

   Srinivasa outlines a reinforcement learning model in which a robot acts, observes the world, and updates its plan based on rewards. With a person present, the robot must also infer that person's goals from incomplete observations. He compares this to steering a paddleboard while shaping the water: robot actions can influence later collaboration, not just produce an immediate result.

4. 17:04

   Helping people control robots

   For people with spinal cord injuries, controlling a six-degree-of-freedom robot arm through a joystick or brain-computer interface can be difficult. Srinivasa's team aims to infer the user's intended task and assist with the movement, rather than simply relay each command. A hindsight optimization method can delay commitment when several goals remain plausible, helping across more than one possible target.

5. 22:04

   Feeding assistance and a shared research agenda

   Srinivasa says patients identified robotic feeding as an important need, but a useful system must handle both deformable food and the timing of human interaction. His team uses facial cues, gestures, and gaze to estimate when a person wants help. He closes by calling for a unified mathematical model of interaction that connects prediction, influence, and control.