Ep#54: MemER: Scaling Up Memory for Robot Control via Experience Retrieval

In this talk

1. 1:05

   A hierarchical policy for long tasks

   MemER pairs a fine-tuned Qwen 2.5-VL high-level policy with a fine-tuned π₀.5 low-level policy. The high-level policy selects subtasks and saves informative key frames; the low-level policy carries them out. The team demonstrates tasks such as finding objects hidden in bins, counting scoops, and dusting shelves before returning plushies to their original places.

2. 3:49

   Selecting and using key frames

   The high-level policy sees the eight most recent frames, sampled at two hertz, and nominates candidate frames to retain. An online clustering step merges repetitive nominations into a persistent key-frame bank, keeping recent context bounded while preserving information from much earlier in a task. The team trains frame selection using 50 demonstrations per task, subtask labels, and manually reviewed informative subtask endings.

3. 9:25

   Retries and intervention data

   The key-frame policy is trained to avoid adding redundant memories during retries, while the low-level policy handles retries from the current observation. This separation lets the team train recovery behavior with targeted intervention data: about 15 interventions per task, collected from common failure states. Examples include retrying a scoop and recovering a plushie or shoe that was dropped or stuck.

4. 17:33

   Memory formats and limits

   The team says image-only key frames performed better than versions that added text, possibly because text encouraged the model to memorize the demonstrations' subtask order. They note that snapshots can miss motion or spatial context, and that the current memory buffer cannot delete outdated frames. They identify multimodal memory, memory deletion, and testing generalization across more tasks as open directions.

5. 23:47

   Evaluation and future directions

   The team reports that off-the-shelf GPT-5 and Gemini Robotics models performed worse than their fine-tuned Qwen model in offline comparisons, and their roughly 15-second inference time prevented real-world coordination with the low-level policy. Training one model across three tasks showed some positive transfer to new object and task combinations. The discussion also raises shared memory between robots and a combination of dense short-term context with sparse long-term memories.