Elvis, co-founder and CTO of Mimic Robotics, discusses video-action models and robot learning with the RoboPapers hosts.
Elvis says video models learn visual dynamics and physical motion, while vision-language-action models rely more directly on labeled robot actions. Mimic’s video-action approach adds an action decoder to a video model, aiming to make better use of human video and reduce dependence on teleoperation data.
Mimic uses teleoperation to collect task data, but sees human data as the path to scaling pretraining. Elvis argues that a robot hand modeled on human morphology can transfer human manipulation strategies more readily than an unfamiliar embodiment. Wearable gloves with position sensors, wrist cameras, and planned tactile sensing are one way to collect action-labeled data without operating the robot.
On LIBERO and Bridge benchmarks, Mimic reports that its approach performed well with about 500 episodes, and that 2% of the action data could approach the performance of a Pi 0.5 VLA trained on all the data. In an oracle simulation experiment, providing the action decoder with the ground-truth future video produced 100% success, suggesting the decoder could translate a good visual plan into actions. Elvis cautions that video models can cost more to train, and that industrial behavior absent from pretraining data will not appear simply because the model improves.
Mimic has paying industrial customers, and Elvis says deployment reveals needs that lab benchmarks miss, including failure detection, recovery, and escalation to a teleoperator. He sees reinforcement learning post-training and reward models as important for improving deployed systems, while noting that robotics lacks a settled approach. He favors on-device inference for reliability in factories, followed by distillation to fit capable models onto practical hardware.
Elvis sees simulation as useful for regression testing, but is skeptical that it can supply most pretraining data or reliably transfer manipulation policies to real settings. Mimic plans to scale wearable data collection, develop more native video pretraining, expand deployments, and publish work on reinforcement learning post-training. He also points to trajectory prediction alongside video as a promising pretraining objective.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
Scott Walter, a robotics engineer and advisor, joins Over The Horizon hosts and guests to discuss Sharpa’s Dairy Queen deployment, Mimic’s dexterous hand, and Sunday Robotics’ home laundry results.
Elvis Nava, co-founder and CTO of Mimic Robotics, discusses dexterous robot hands and learning-based manipulation with host Devang on The Humanoid Hub.
