Russ Tedrake, MIT associate professor and Toyota Research Institute vice president of robotics research, discusses feedback control and manipulation at a University of Toronto Robotics Institute seminar hosted by Florian.
Tedrake wants manipulation to gain the robustness that model-based feedback control brought to humanoid robots in the DARPA Robotics Challenge. He argues that controlling an arm is comparatively easy; the harder problem is controlling objects and the changing world around them. Passive, underactuated grippers work well for some tasks, but cannot match the range of human hand movements.
Tedrake explores how robots can represent objects without relying on a single canonical pose, which may not fit every item in a category. Human-labeled semantic keypoints, such as a shoe’s toe and heel, help define tasks like placing shoes on a rack, while point clouds and shape completion support grasping. Dense visual correspondences can be trained from reconstructed objects and used to identify corresponding regions across different instances.
Tedrake describes imitation learning that combines dense visual descriptors with a compact feedback policy. The approach handled tasks such as flipping shoes and pushing a plate, including changes in appearance and object dynamics. He also discusses learning keypoint dynamics for model predictive control, which could use a demonstration to specify a task, though planning with learned models remained difficult.
For pushing carrot pieces into a target area, Tedrake studies a policy that maps camera images to actions and can handle different numbers of pieces. A learned image-prediction model sometimes produced planning failures, so the team tried a linear image transition model that discouraged carrots from disappearing. It performed robustly in this setup, a case Tedrake connects to pixels behaving approximately independently.
Tedrake distinguishes compact state-space models from image-based input-output models, which can avoid the burden of estimating physical state and parameters. He says manipulation representations must fit the task, and that tactile sensing matters because cameras lose useful views near contact. In the discussion, he argues that robustness should account for varied objects and environments, while broad uncertainty can make demanding tasks impossible.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
MIT robotics professor Russ Tedrake discusses physical AI, robot learning, deployment, and the human impact of automation with Automated host Bryant Heater in Boston.
MIT professor and Toyota Research Institute vice president Russ Tedrake explains how graphs of convex sets can combine trajectory optimization and discrete search for robot planning at the MIT Robotics Seminar.
