Harold Soh, who leads the Collaborative Learning and Adaptive Robots (CLEAR) Group, discusses ways to connect robotics with generative AI at an IEEE Robotics and Automation Society event.
Soh’s group works on assistive robotics, including tasks such as feeding and oven cleaning, where robots must manage both physical actions and human interaction. He sees promise in more capable, lower-cost robots and generative AI, while noting that many robots still rely on teleoperation and generative models often lack reliable physical understanding.
Diffusion policies turn noise into action sequences, but can require many inference steps to produce reliable movements. Soh’s Bridger method starts from a more informative distribution, such as a heuristic, planner, or learned policy, then refines it toward the target using interpolation. In grasping and synthetic wound cleaning, this approach improved results with few steps; grasping performance exceeded 80% at 10 or 20 steps.
Soh’s group connects tactile sensor data with language so models can describe properties such as hardness, roughness, and bumpiness. Its dataset began with 70 objects, roughly 400 videos, and about 2,000 annotations, and the team later expanded it substantially. The Octopi model described toothbrush bristles and handles from touch data and used tactile cues to select a softer, riper avocado, though ripeness accuracy was about 63%.
Soh says tactile language models are early work and need stronger datasets, encoders, and overall performance. He sees touch as both complementary to vision and useful for properties that are difficult to express in words. For robotics more broadly, he calls for architectures that combine sensing, task planning, and motion planning, with careful study of when transferring knowledge helps or hurts.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
Harold Soh, a robotics researcher from Singapore, discusses a proposed foundation model for human-robot interaction at an IEEE Robotics and Automation Society event.
Harold Soh of the National University of Singapore discusses robot learning and human-robot interaction at the Fall 2022 GRASP SFI, hosted by GRASP Lab.
