Shiao-Ju, who works on tactile sensing and related algorithms at Sharpa, presents at GTC 2026 on using tactile AI for high-degree-of-freedom dexterous manipulation.
Shiao-Ju proposes two stages of action: vision guides the hand into position, then tactile and proprioceptive feedback guide contact movements. The architecture divides work across System 2 for task reasoning, System 1 for coarse motion, and System 0 for real-time feedback control. System 0 uses the smallest model and targets 100 Hz control to manage computational cost.
Sharpa combines its System 1 and System 0 in CraftNet, with a vision-language model providing subtask descriptions. System 0 estimates a distribution of small action changes conditioned on tactile sensing, proprioception, and intent. Shiao-Ju says short observation and prediction horizons, fewer solver steps, and a small model help speed inference.
The proposed data pipeline starts with basic actions such as grasping, pinching, insertion, and in-hand rotation. Simulation and reinforcement learning generate pre-training data, while real-robot data supports later fine-tuning and longer tasks. In Isaac Sim, the team models tactile forces, contact points, and deformation maps; reported comparisons show contact-point error below 1 mm, force error around 0.5 N, and deformation-area overlap above 85%.
Shiao-Ju reports System 0 inference at 400 Hz on an RTX 4060 GPU, alongside tactile sensors running at 180 Hz. In demonstrations, tactile feedback helps a hand rotate a ball and roll it between two fingers despite noisy teleoperation inputs; direct teleoperation without System 0 often drops the ball. The remaining challenges include better modeling of nonlinear hardware behavior and generating useful physical interaction data.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
TAKASU Masakazu, host of Nico-Tech Shenzhen, reports from ROSCon China 2026 in Shenzhen on the robotics demos, developer tools, and embodied AI themes across the event.
A Sharpa presenter introduces the company’s dexterous robot hands, CraftNet AI model, data strategy, and real-world deployments.
A Google DeepMind main speaker explains how Gemini Robotics 2 aims to help robots handle varied physical tasks, including coordinated movement, dexterous manipulation, and teamwork between robots.
Scott Walter, a robotics engineer and advisor, joins Over The Horizon hosts and guests to discuss Sharpa’s Dairy Queen deployment, Mimic’s dexterous hand, and Sunday Robotics’ home laundry results.
