AI in the physical world: Intelligence beyond the screen
Professor K. Madhava Krishna of IIIT Hyderabad explains how robotics can combine foundation models with classical planning, drawing on research at the Robotics Research Center; a panel hosted by TCS Research and ACM India explores the wider prospects for physical AI.
In this talk
- 19:30
Why robotics challenges AI models
Krishna describes a push to bring the success of language and vision models to robots that can work across settings and embodiments. But robotic systems must turn images and instructions into physical actions while meeting constraints such as collision avoidance, smooth motion, and joint limits.
- 22:40
Combining learned models with classical methods
He traces robotics from handcrafted perception and classical planning through deep learning and generative AI. Language models make it easier to give robots instructions, while generated data and candidate trajectories can help with planning. Krishna argues that learned systems still struggle with hard physical constraints, so his group combines them with tools such as model predictive control and search.
- 32:00
Examples from mobility and manipulation
Projects at IIIT Hyderabad include a wheelchair that interprets open-ended requests, identifies objects in 3D, and plans routes around people, as well as a driving system that uses language and visual models to identify landmarks. For manipulation, the group applies optimization to learned motion plans to reduce collisions. Krishna’s conclusion is that different tasks call for different combinations of methods.
- 52:25
What physical AI needs to mature
Panelists describe physical AI as systems that sense and act in the real world, while noting that the term is broad and robotics has a long history. They expect robots to change many jobs and assist with dangerous or repetitive work, but say reskilling and thoughtful planning matter. Panelists also point to hardware cost and dexterity, scarce training data, privacy, and the limits of simulation as major challenges.
- 1:13:44
Building practical and adaptable robots
Panelists call for stronger hardware development, grants that account for product failures, and more product design in engineering education. Simulation is valuable for training and testing, but becomes less reliable as tasks involve richer contact, manipulation, people, and pets. General-purpose robots remain out of reach; practical progress depends on adapting systems to particular settings and combining methods carefully.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.