[ICON Spring26 Seminar] Tingnan Zhang (Google DeepMind) #ai #robotics #gemini

In this talk

1. 5:05

   What general-purpose robots need

   Zhang defines five goals: dexterity, broad task coverage, safe interaction, physical reasoning, and the ability to carry out longer tasks from high-level requests. He wants robots to adapt to unfamiliar tasks with limited additional data, while noting that current systems remain far from that goal.

2. 14:45

   One model across different robots

   Zhang describes Gemini Robotics 1.5 as a single pretrained model for platforms including ALOHA, a dual-arm setup, and a humanoid. He says it can transfer some skills between embodiments, including tasks collected on one platform and tested on another, though performance may be less smooth in a new setting. The team collects each platform’s demonstrations through its own teleoperation setup.

3. 31:08

   Combining action with embodied reasoning

   The action model can produce language that guides its own movements, which Zhang says helps on nuanced tasks. A higher-level reasoning model can estimate object states, track task progress, and coordinate actions over longer sequences, such as choosing safe items for an allergy-aware lunchbox. He reports improved spatial and object detection accuracy in ER 1.6, with some measures moving from the 80s into the upper 90s.

4. 51:31

   Open problems in data, learning, and evaluation

   Zhang identifies data volume and quality as central challenges, while noting that teleoperation limits collection and the field has not settled how much data is needed. He says reinforcement learning with large models remains difficult to stabilize, and robotic progress is hard to measure because dataset losses may not reflect closed-loop success. Simulation can help compare models if it preserves their relative ranking on real hardware.

5. 1:02:37

   Research questions and system design

   Zhang discusses world models, compute needs, and how sensing and control might be organized. He suggests fast force and torque responses belong in lower-level controllers, while tactile input may fit into larger models depending on its representation. In Q&A, he says the robotics models are served from the cloud, and that supporting new hardware requires engineering work.