Krishna Murthy on ∇SLAM and ∇Sim | AI in Robotics Seminar
Krishna Murthy, a PhD candidate at Mila’s Robotics and Embodied AI Lab, discusses differentiable programs for SLAM, simulation, and robotics at the AI and Robotics Seminar hosted by the University of Toronto.
In this talk
- 2:10
Structured, differentiable robotics programs
Murthy argues that robotics can retain classical structure while using modern learning methods. He proposes representing modules such as perception, planning, and control as structured, differentiable programs, with the aim of improving performance, interpretability, and generalization.
- 6:26
GradSLAM connects 3D errors to image pixels
GradSLAM makes dense SLAM components differentiable so errors in a 3D map can send gradients back to sensor pixels. Murthy shows examples of identifying uncertain image regions and refining depth estimates, even when the initial depth comes from a noisy estimate. The approach needs 3D supervision, and long unrolled computations limit it to a few hundred frames in the examples described.
- 26:00
GradSim learns from rendered images
GradSim combines a physics engine with a renderer, allowing optimization from target images or videos rather than tracked positions and velocities. It uses an adjoint method to backpropagate without storing every intermediate simulation state. Murthy demonstrates estimating physical properties such as mass, friction, and elasticity from video.
- 46:10
Visual goals guide simulated motion
Murthy describes optimizing a simulated gear, cloth, and 2D walker using image-based objectives. For the walker, a neural network adjusts actuation so its final image matches a target; the demonstration begins walking toward the target after one optimization iteration and reaches the target configuration after two. Results depend on the simulator’s physical assumptions being reasonably close to reality.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.