Harold Soh of the National University of Singapore discusses robot learning and human-robot interaction at the Fall 2022 GRASP SFI, hosted by GRASP Lab.
Soh describes the shift from robots working in structured settings toward tasks around people, including assistive wheelchairs and close-contact care. He argues that robots need both physical abilities, such as tactile sensing and safe control, and social abilities that help people feel at ease.
Soh introduces latent state-space models as a way to represent hidden states, observations, actions, rewards, and changing environments. Neural networks make these models more expressive, while variational inference helps train them and the probabilistic structure provides uncertainty estimates useful for safety.
Soh presents MuMMI, a method for learning shared representations from modalities such as video, depth, and tactile sensing, even when inputs are unreliable or arrive at different rates. In simulated tasks including walking, finger spinning, and cup catch, the approach outperformed comparison models; a simulated spill-cleaning robot also continued working when a sensor was removed.
Soh describes a model that simulates a person's possible actions and plans what information a robot should share to help them complete a task. Small perception and policy implants capture differences between the robot and human models. In a foggy driving study, participants using the planning approach had fewer collisions and rated its communication as more helpful and timely than behavioral cloning.
Participant feedback showed that some people valued a more talkative agent because its frequent updates made it seem more dependable, even when they found it annoying. Soh says future models should account for trust and for situations where the human and robot each have only partial or different information.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
Harold Soh, a robotics researcher from Singapore, discusses a proposed foundation model for human-robot interaction at an IEEE Robotics and Automation Society event.
Harold Soh, who leads the Collaborative Learning and Adaptive Robots (CLEAR) Group, discusses ways to connect robotics with generative AI at an IEEE Robotics and Automation Society event.
