CoRL 2024 Day3 14:00-16:00
Vikash, MyoLab co-founder and Carnegie Mellon adjunct professor, presents an early-career keynote at CoRL 2024 on robust robot learning, data augmentation, and universal agents.
In this talk
- 6:43
From optimality to robustness
Vikash argues that real robots rarely have perfect state estimates or accurate models, so optimizing for a single ideal behavior is brittle. He proposes policies that stay within a wider distribution of states and actions, using observations directly and aiming for robust behavior rather than an optimal point.
- 13:15
Use generative augmentation to widen training data
Vikash describes augmenting clean tabletop demonstrations with generated changes to objects, textures, backgrounds, and distractors. With 10 demonstrations, state-only augmentation helped a pick-and-place policy generalize to varied scenes. He says pretrained models contribute world knowledge that can expand the training distribution while keeping the demonstrated action valid.
- 21:09
Build useful representations before tasks arrive
Vikash compares robot learning to getting dressed: much of the preparation happens before the final task. He advocates learning from simulation, human data, and other off-domain sources, then grounding that knowledge with smaller amounts of robot interaction. He frames this as a representational MDP, where foundation models provide structure for observations, goals, dynamics, and rewards.
- 28:22
A small dataset for a multi-skill agent
Vikash presents RoboSet, a dataset of 7,500 trajectories across 12 skills, and a language-conditioned policy tested in hundreds of unseen situations. The team uses generated semantic image augmentations to expand its data without additional human or robot demonstrations. He reports stronger generalization than RT-1 and BET in the described comparisons, and says performance improved as the number of augmentations increased.
- 33:50
Good enough hardware and a path to viability
Vikash argues that robust policies and learned invariances can make repeatable, high-end hardware less essential. His closing advice is to seek minimum viability by combining useful skills and representations, rather than building capabilities one small piece at a time. In Q&A, he suggests 10 to 15 skills may cover many home tasks, while noting that tasks with complex action distributions need more demonstrations.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.