Ep#69: MolmoSpaces, an Open Ecosystem for Embodied AI

In this talk

1. 1:25

   A large collection of indoor scenes and objects

   Yujin describes 230,000 indoor environments spanning one to ten rooms, about 130,000 objects, and millions of annotations. The five scene datasets include handcrafted, converted, procedurally generated, and LLM-guided environments, covering settings such as kitchens, arcades, and offices. The team prepared scenes for MuJoCo and provided compatibility with ManiSkill and iGibson.

2. 12:35

   Grasps and reusable manipulation tasks

   Omar explains how the project generates and checks roughly 1,000 grasps per object. For articulated assets such as doors, the team tests whether a grasp on the handle allows the joint to open and close. The task library includes picking, placing, opening, closing, and navigation, with rewards that track progress as well as final success.

3. 25:23

   Building and testing a diverse benchmark

   The team creates benchmark episodes by randomizing scenes, objects, robot positions, and camera positions, then filtering for configurations a scripted solution can complete. They balance the set across objects and environments and provide configurations as downloadable JSON. Tests of open-source policies include 1,000 to 2,000 distinct episodes, and reported simulation results correlate strongly with real-world picking results from RoboArena.

4. 31:40

   Policy results and distribution shifts

   Omar reports that picking success improved across π policy versions, from 14% for π0 to 28% for π0.5 in the cited comparison. Experiments also show that performance depends on how closely prompts and robot setup match training conditions; changing camera inputs or adding noise to spatial conditioning can reduce success. The team presents simulation as a way to run these stress tests at scale and complement costly real-robot evaluations.

5. 54:47

   An open ecosystem with room to extend

   The researchers describe the release as a starting point for loading scenes, assets, and annotations, evaluating policies, and building custom tasks. They note limits, including that most objects are rigid and only a smaller set is articulated, and say they plan to release training data and more infrastructure. They invite contributors to add robots and objects, and see multi-room scenes as useful for future mobile manipulation work.