RoboBoom
GRASP Lab· November 11, 2022· 1:02:02

Fall 2022 GRASP SFI Roberto Calandra, Meta AI

Roberto Calandra, a researcher at Meta AI, presents his work on tactile sensing at the GRASP Lab’s Fall 2022 SFI seminar in Philadelphia.

In this talk

  1. 0:31

    Why robots need better interaction

    Calandra argues that robots need to adapt to unpredictable settings, since scripted or remotely operated systems can fail at tasks people find simple. He points to industrial automation, disaster response, exploration, and elder care as areas where capable robots could help.

  2. 8:28

    Building a reusable touch ecosystem

    Tactile sensing has been difficult to advance because it spans hardware, data processing, and control, while many research sensors remain one-off prototypes. Calandra describes standardization as a way to lower the cost of entry and help researchers reuse tools. The ecosystem he presents includes DIGIT hardware, the Capa simulator, and PyTouch software.

  3. 15:28

    Making tactile hardware and simulation accessible

    Calandra describes DIGIT as an open-source, camera-based tactile sensor that can connect over USB and capture fine surface detail. He compares its roughly 15-dollar materials cost with BioTac at about $15,000, and says more than 20 universities had replicated the design. Capa simulates vision-based tactile sensors at hundreds of frames per second and can support sim-to-real learning.

  4. 28:09

    Processing touch and measuring progress

    Calandra presents PyTouch as a machine learning library with sensor interfaces and pretrained models, intended to make touch experiments easier to run. He describes work on reusable representations learned across sensors, and calls for shared benchmarks and datasets. One dataset contains a million grasps on a single object.

  5. 35:03

    Applications and the open question of necessity

    Potential uses include prosthetics, tactile search for online shopping, robotics, and remote touch. Calandra highlights fine manipulation of a marble between robotic fingers using tactile readings and model-based reinforcement learning. He also reports that some simulated manipulation policies transferred to robots using only joint position and velocity, so the case for touch as essential to manipulation remains unresolved.

Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.

More talks