2024 Winter Robotics Colloquium: Liyiming Ke (Allen School)

In this talk

1. 2:28

   Why fine manipulation is hard

   Ke focuses on tasks such as grasping coins and handling small food items, where errors under a millimeter can matter. Factory systems can assemble intricate products such as watches, but moving those capabilities to homes or other uncontrolled settings is difficult.

2. 8:36

   A chopstick robot and learning from demonstrations

   Ke describes an accessible robot built from hobby actuators and 3D-printed parts, with chopsticks as its gripper. Thirty campus participants teleoperated it successfully, though actuator backlash can produce up to 5 mm of tip error and simulation still differs from real-world behavior. Ke explains how imitation learning can fail on multimodal demonstrations and how continuity-based data augmentation can generate corrective labels near demonstrated states.

3. 30:14

   Adapting beyond a fixed dataset

   Ke argues that difficult tasks often lack enough high-quality demonstrations, and changing starting conditions make exhaustive data collection impractical. In a human study, allowing retries raised success on the hardest ball-grasping task from 26% to 80%. CherryBot combines simulation pretraining, demonstrations, and real-world reinforcement learning to adapt through trial and error; it reached about 64% success across varied tasks.

4. 43:04

   Structure, deployment, and human dexterity

   Ke says data-driven robotics can benefit from physical structure and prior information, including local continuity, approximate simulators, and imperfect demonstrations. She describes future work on shared autonomy and collecting data in homes, with systems that can assist even when human intent or behavior is uncertain. Her long-term aim is to help people facing age-related loss of manual dexterity.