2026 Winter Robotics Colloquium: Florian Shkurti (University of Toronto)

In this talk

1. 3:55

   Off-policy evaluation with STITCH

   Shkurti describes STITCH, which estimates a target policy’s value from logged behavior data without running the target policy in the real world. It uses a diffusion model, trajectory chunking, and positive and negative guidance to generate samples closer to the target policy. In experiments on D4RL environments, he reports strong results and an estimator variance bound that grows quadratically with episode horizon, while noting that the method needs overlap between behavior and target data and a valid reward model.

2. 16:17

   How diffusion planners compose trajectories

   Shkurti and his student Quentin study why diffusion planners can combine parts of trajectories they saw during training. Their experiments suggest that local receptive fields and shift equivariance can improve composition when data is scarce, while larger datasets can lead different architectures to learn locality on their own. The practical choice depends on whether composition is needed and how much training data is available.

3. 26:23

   Evaluating policies inside video models

   Shkurti describes using an action-conditioned video model to simulate policy rollouts, then asking a vision-language model whether the task succeeded. Results in simulation show promising agreement with ground truth, but the approach can hallucinate objects, lose consistency across views, or repeat predictions when actions are small. It relies on post-training for the environment, and real-data results are weaker than simulated ones.

4. 40:52

   Generating adversarial scenarios for robustness

   Shkurti discusses creating scenarios that reveal policy failures and could transfer from simulation to the real world. In driving experiments, optimized 3D objects caused deviations, but he says realism and transfer remained unresolved. For manipulation, his group uses Gaussian splatting to edit scenes with scanned objects, lighting, and backgrounds; iterative adversarial retraining improved robustness to relighting in Push-T, though the work is ongoing.

5. 54:17

   Connecting evaluation and data collection

   Shkurti identifies a longer-term goal of using simulation and policy evaluation to guide real-world data collection, while using collected data to improve simulation. He also points to open work on evaluating policies from states or video alone, without robotics data, where current assumptions may not hold.