Introducing S1: A robot model that learns from one example

In this talk

1. 0:07

   Learning from one video

   The presenter says people can learn from a single example and describes S1 as a robotics model that can pick up new tasks from one video. The model uses the demonstration as context to produce robot actions, including for tasks outside its training data.

2. 1:03

   Long tasks and new skills

   S1 can retain the steps of demonstrations lasting up to 10 minutes, with examples including making pancakes, preparing coffee, and potting a plant. After it flipped a pancake, the team reviewed roughly one million hours of pre-training data over two days and found no pancake-flipping example. The presenter says S1 inferred the action from the spatula's movement in the video.

3. 2:21

   Generalizing to unfamiliar setups

   For tasks represented in pre-training, the presenter says in-context learning performs comparably to language-prompted vision-language-action models. It is especially useful for unfamiliar tasks, such as assembling a hygiene kit with specific contents and steps after seeing the demonstration shortly before deployment. S1 can also adapt when objects shift, recover from mistakes, and infer the intended outcome when a demonstration includes errors.

4. 3:47

   Lowering the cost of automation

   The presenter says performance declines gradually as prompts, object positions, and objects change, so one recorded demonstration can transfer across locations. Matching S1's accuracy with current VLA models would require an estimated 50 to 100 hours of data collection, compared with one example for S1. The presenter envisions this making robotics accessible to smaller businesses and organizations.