GTC SJ 2026: Physical AI for Healthcare Robotics - Simulation-First Design & Accelerated Development

In this talk

1. 0:05

   The healthcare robotics data pipeline

   Mustafa Jay frames healthcare as a high-demand field where physical AI could help, while emphasizing that data is the main challenge. He outlines a pipeline of curated real-world data, synthetic data and simulation to fill gaps, model training, and testing. He highlights generative physics, foundation models, and faster, higher-fidelity simulation as technologies that could support this work.

2. 5:32

   Surgical data and simulation

   Filipo Fukori, a surgeon and robotic fellowship program director at Lenox Hill and Northwell Health, describes barriers to using surgical data, including medical-legal issues and the time required for clinician annotation. He says important events may occupy only 2% to 3% of a procedure video, and describes work on surgical gesture ontologies and using Cosmos tools to search and simulate surgical scenes. He also argues that simulation is underused in surgical training and describes efforts to create more realistic tool and tissue interactions.

3. 12:47

   Learning and testing autonomous surgery

   An associate professor at Johns Hopkins who also identifies himself as chief robot officer at CMR Surgical describes a shift from model-based methods, which reached about 60% stitch-to-stitch success, toward imitation learning from demonstrations. His examples include tasks that achieved 100% success in reported trials, and an autonomous clipping and cutting phase tested on ex vivo porcine gallbladders using about 16 hours of data. He also discusses a surgical robotics dataset with more than 150,000 trajectories and early GR00T suturing results.

4. 22:06

   Simulation for robot setup and deployment

   Mustafa Jay describes Maestro, a two-armed laparoscopic system, and a simulation workflow for optimizing arm deployment across different operating room setups. The team combines real video, Isaac Sim, and Cosmos Transfer to generate varied training imagery, while noting that the synthetic visuals still need improvement. Jay says the intended first step is a surgeon-triggered workspace optimization feature, with later work exploring vision-language-action models under safety controls.

5. 30:43

   Hard-tissue robotics and collaboration

   Yossi, founder and CEO of LEM Surgical, contrasts hard-tissue robotics with soft-tissue systems and presents a three-arm architecture for orthopedic procedures. He says the robot must learn to work around surgical staff and many kinds of instruments, a problem he considers too complex for fixed decision matrices to scale. He sees simulation and synthetic data as ways to train the system before deployment in the operating room.