Softening the boundaries: the human, the physical, and the computational
A University of Michigan computer vision researcher and founder of an open source vision company describes how his work aims to connect human understanding, physical environments, and computation to build systems that can team with people.
In this talk
- 0:00
A career shaped by people and early technology
The speaker credits family, mentors, and collaborators, including adviser Greg Allen, haptics researcher Allison Okamura, and Michigan faculty member Jeff Fessler, with shaping his career. His parents’ decision to buy a Commodore 64 in the mid-1980s helped spark his interest in computing.
- 5:01
From computer vision to interactive systems
He says computer vision has supported his career, including an open source computer vision company with millions of downloads and a team of more than 60 people across North America. But his broader ambition is to create systems that work alongside people, an idea sparked by childhood hopes of playing with Nintendo’s R.O.B. robot.
- 8:20
Connecting perception, language, and physical action
His research spans multitouch interfaces, brain imaging, image captioning, and robots that interpret their surroundings. Projects included a PackBot that could identify stairs it could climb, and image captioning work that combined object detection, tracking, scene relationships, and language to describe video.
- 13:06
Letting people shape what systems learn
The speaker argues that people should help determine what a machine learning system is trying to achieve. In active clustering, human feedback changes how examples are grouped, such as distinguishing dog breeds, colors, or individual dogs depending on the task.
- 17:53
Building AI guidance for real tasks
Robot Slang paired a map-reading commander with a robot-driving partner who communicated through text, letting them develop task-specific language in a maze. Later projects included Millie, an augmented reality agent for guiding cooking, and Vigil, which projects procedural guidance to help generalist clinicians perform tasks such as cardiac ultrasound amid physician shortages.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.