Learning to Design Robots -- Yi Sha

In this talk

1. 0:35

   Why hardware and control need to be designed together

   Robot development typically moves from mechanical design to manufacturing, control, and real-world trials. When a robot fails, the cause may lie in its shape, its policy, or both. Changing hardware can also mean rebuilding control and data from scratch, making manual iteration slow.

2. 7:55

   Learning a soft gripper for varied objects

   Sha describes an end-to-end system for jointly learning a grasping policy and the design of a tendon-driven soft finger. The team varied stiffness and tendon routing, then trained a differentiable neural physics model on about 80,000 simulated outcomes. The learned gripper was asymmetric: a firmer finger supported an object's weight while a more flexible one conformed around it.

3. 15:53

   Using human motion to shape robot hands

   Sha argues that design and control have different roles at deployment: hardware stays fixed while control can change. A second project uses human motion data from OakInk to guide hand design, including joint layout and finger movement. The team also explored 3D-printed linkage fingers with living hinges to reduce the need for motors.

4. 21:49

   Searching designs with learned proposals

   Because gradient optimization can struggle with linkage parameters, the team trained an encoder and actor to propose candidate designs and controls, then used local gradient descent to refine them. For a repetitive motion, the learned actor found a suitable design in about half an hour, compared with five hours for a sampling-based method. Sha notes that the motion demonstrations were replayed without physics.

5. 27:23

   What robot design research still needs

   Sha says automated robot generation remains far off and points to three needs: expressive yet physically plausible design spaces, simulations that rank designs reliably against real-world results, and control methods that adapt across different embodiments. He observes that hardware rankings may transfer from simulation even when the policies themselves do not.