AI for Humanoid Robotics - A lecture by Mentee Robotics' CEO, Prof. Lior Wolf
Mentee Robotics CEO Prof. Lior Wolf explains the company’s approach to building humanoid robots that combine learned skills, perception, navigation and language based planning.
In this talk
- 0:00
Why robotics remains a hard AI problem
Wolf argues that general purpose robots are a major challenge because they must learn through physical experience, which is slow and can be unsafe to collect. Simulation offers abundant training data, but differences between simulated and real physics make transfer to real robots difficult. He points to language models, vision foundation models, semantic 3D mapping and onboard compute as recent advances that can help.
- 3:08
An integrated, compound robot system
Mentee develops both its AI and robot hardware, combining reasoning, locomotion, navigation and dexterity. In a kitchen demonstration, the robot follows spoken instructions, navigates to a table, handles fruit and places a container on a counter. Wolf says the demonstration integrated roughly 20 to 30 models, with speech recognition and speech generation among the few components sourced externally.
- 12:25
Why simulation and corrective policies matter
Wolf favors compound AI systems that connect specialized components, arguing that general purpose robots face too many tasks, environments and movement possibilities for an end to end model to cover with real world data alone. Mentee trains locomotion policies with reinforcement learning in simulation, then uses real world movement data to learn action corrections that improve the simulator’s match to reality. The resulting policy is retrained in simulation and deployed without the correction term.
- 30:00
Perception and navigation in changing spaces
The robot’s software connects real time motion control, perception and a language model that turns requests into API calls. Navigation combines a static semantic 3D map with dynamic obstacle information from stereo vision, then uses separate policies for efficient travel and precise final positioning. Wolf also describes using NeRF based maps, visual tracking and open vocabulary queries to locate the robot and find objects.
- 41:20
Capabilities under development and safety
Wolf outlines planned work on onboarding in new workplaces, including following a person, learning places and objects from pointing, and building maps onboard. The team is also working on manipulation such as pressing buttons and opening doors, while event triggered agent behavior and person reidentification remain areas to develop. A fall detection model helps the robot recover when possible or soften an unavoidable fall to protect people and equipment.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.