Andrea Bajcsy, a robotics researcher at Carnegie Mellon University, discusses how to make increasingly capable generalist robots safer, with examples from her lab’s work at the GRASP seminar at Penn.
Bajcsy argues that robots handling varied physical tasks need safety concepts beyond collision avoidance and force limits. A kitchen robot, for example, should recognize risks such as burning food, spilling hot oil, or tearing an object while still being able to handle fragile items.
Handwritten state and dynamics models become unwieldy for tasks such as lifting an open candy bag, where contents can shift and spill. Bajcsy’s group learns latent world models from images, trains a classifier to recognize failure, and uses reinforcement learning to approximate a safety filter. In a demonstration, the filter slowed a teleoperator’s shaking and blocked a pull likely to spill the candy; it prevented spills for 17 of 20 people in a public stress test.
The learned filter can mistake an upside-down open bag for a closed one when cameras cannot see its opening, and a model trained on one type of bag may not transfer to different materials. Multimodal training can make image-based representations more useful for heat constraints: predicting infrared data during training helped a robot avoid overheating wax at test time. For unfamiliar outcomes, the group uses model disagreement and conformal prediction to flag states the robot cannot reliably predict.
Because partial observability can produce several outcomes from the same action, Bajcsy’s group developed Stress Test to search video world models for plausible, damaging possibilities. It uses a vision-language model to score generated videos and constrains optimization to remain in the model’s typical noise region, reducing implausible results. The group also used these searches to reweight demonstrations before fine-tuning a robot policy.
Bajcsy describes test-time scaling along two dimensions: taking more precise denoising steps and evaluating multiple action samples. Her group learned a meta-policy to allocate that computation based on the robot’s state. On hardware with π0.5, it improved performance by 20% while reducing inference latency by 34%.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
Andrea Bajcsy of Carnegie Mellon University gives the 2026 Spring Robotics Colloquium at the Paul G. Allen School, discussing how to make generalist robots safer.
