Fall 2024 GRASP SFI - Dian Wang, Northeastern University

In this talk

1. 0:02

   Why symmetry can save training data

   Wang frames sample efficiency as a practical problem: prior robot learning work collected 800 robot-hours, while small labs may only have a few hours to train. Her approach uses known symmetries, such as rotating a scene and its corresponding action, so the robot can generalize across related situations.

2. 5:17

   Equivariant reinforcement learning

   Wang describes group-equivariant Markov decision processes, where rotating the state and action preserves transitions and rewards. She uses equivariant networks in actor-critic and Q-learning methods, reporting faster learning than standard CNN baselines. In real-robot trials, several tasks took 45 minutes to an hour of training; block-in-bowl took about 2 hours and 40 minutes.

3. 24:57

   From Q-learning to diffusion policies

   For spatial Q-learning, the network assigns values to possible grasp positions and orientations, with rotations transforming both the map and its orientation channels. Wang then applies symmetry constraints to diffusion policies, which learn to denoise action sequences from demonstrations. In 12 simulated tasks, the method improved results over several baselines, and 200 demonstrations matched or exceeded baseline performance with 1,000.

4. 43:41

   Learning from a small number of demonstrations

   Wang describes applying the approach to robot actions represented as 3D gripper poses while using discrete rotational symmetry. Real-world examples include a task trained from six demonstrations and several others using fewer than 50. She also reports improvements on tasks with varied object placements, with the largest gains on environments whose variation better matched the symmetry built into the model.

5. 48:26

   Working with imperfect symmetry

   A camera angle or fixed background can make image rotations differ from the transformations of objects in the workspace. Wang presents results suggesting equivariant models can still help under this mismatch, including tests using third-person images. She identifies broader symmetry types, combining efficient models with foundation models, and extending the methods to navigation and locomotion as future directions.