Interactive Imitation Learning: Planning Alongside Humans (Sanjiban Choudhury, Cornell)

In this talk

1. 8:02

   Why imitation learning can fail

   Choudhury shows how a lane-change model can score 99% on held-out data yet persist after an unsafe initial decision. Because its actions affect the states it later encounters, mistakes can push the robot beyond its training data and compound over time. He distinguishes settings where behavior cloning works, where it can fail, and where interactive methods can help.

2. 19:04

   Learning from interventions

   For settings where demonstrations do not cover the states a learner may reach, Choudhury describes DAgger, which queries an expert on the learner’s visited states. He presents expert intervention learning (EIL) as a way to use both interventions and periods when the human allows the robot to continue. In MuSHR car experiments, EIL improved with interventions and eventually ran without Jonathan taking over; Choudhury says this happened after about 60 seconds of trials.

3. 24:34

   Matching learner and expert experience

   Choudhury describes a middle setting where a human expert is unavailable for ongoing queries but an interactive simulator is available. Methods can match reward moments, value moments, or combine planning with density weighting, trading off computation and simulator samples. He also outlines an Aurora pipeline that recreates intervention events in simulation, updates models, then checks the learned behavior on the road.

4. 30:24

   Planning around human intent

   A forecast model that averages two possible merge behaviors can produce a broad, uncertain prediction, leading the planner to brake heavily. Choudhury’s approach reasons over discrete interaction modes, such as which driver yields, using a graph neural network to infer modes and predict trajectories. He describes its use in merges and intersections, including a production model tested on roads and in simulation.

5. 39:57

   Open questions and discussion

   Choudhury outlines research questions for his Cornell lab, including learning from language, vision, and touch, identifying interaction modes in everyday tasks, and making robots safe and predictable near people. In the discussion, he addresses imperfect planning, remote human assistance, and how the quality of an initial policy may affect the number of interventions needed.