Xin Chen on Representation Learning for Imitation | Toronto AIR Seminar

In this talk

1. 1:46

   Why study representations for imitation

   Imitation learning avoids designing rewards and can require less compute than reinforcement learning, but depends on costly expert demonstrations. Chen’s goal was to improve policy quality without collecting more demonstrations, by testing whether self-supervised representations help.

2. 3:39

   A framework for comparing methods

   Chen introduced EIRL, a modular framework for testing representation learning methods with imitation learning. It covers pre-training and joint training, where a representation loss supports policy learning. The study compared five representation learners across behavior cloning and GAIL, using DeepMind Control, MAGICAL, and CoinRun tasks.

3. 12:09

   Augmentation often explains the gains

   Image augmentation improved behavior cloning in several settings. But when augmented behavior cloning was the baseline, representation learning added little; Chen reported a similar pattern for GAIL, which needed stronger regularization. The augmentations included random crops, translations, and color jitter.

4. 18:14

   Task structure shapes useful features

   Chen argued that sequential tasks differ from image classification: observations requiring different actions can look nearly identical, while classification examples often have clearer visual differences. A SimCLR-pretrained model focused on broad shapes and background color in CoinRun, rather than the agent, enemy, and goal. She recommended checking whether learned features capture what matters for control and evaluating across varied benchmarks.

5. 22:29

   Open questions and conclusions

   The learned embeddings did not cluster cleanly by action; in one VAE example, they aligned more clearly with state values. Chen concluded that augmentation usually helps, while representation learning added limited value in these experiments. In discussion, she noted that results with very small or random demonstration datasets were mixed, and that action prediction or control-informed learning may be worth further study.