2024 Fall Robotics Colloquium: Ching-An Cheng (MSR AI Frontiers)
Ching-An Cheng, senior researcher at MSR AI Frontiers, presents Trace, a framework for optimizing AI agents from rich feedback, at the University of Washington Robotics Colloquium hosted by Abhishek Gupta.
In this talk
- 1:02
Learning from richer feedback
Cheng asks how agents could learn from feedback such as user comments, console errors, and observations from embodied tasks, instead of relying only on hand-designed losses or rewards. Trace aims to automate the human work of interpreting this feedback and updating an agent.
- 10:44
How Trace represents and updates workflows
Trace builds a computation graph from Python functions and objects, including prompts and code, which can be treated as trainable parameters. Feedback travels backward through the graph to an optimizer, which can update parameters without requiring every operation to be differentiable. Cheng describes OptoPrime, which represents the graph as pseudocode for an LLM to inspect and revise.
- 24:42
Experiments across agents and robots
Using the same OptoPrime prompt template across several applications, Cheng demonstrates optimization of plans for collaborating household agents and of both prompts and extractor code on Big-Bench-Hard. In a simulated Meta-World pick-and-place task, feedback helped a Python controller progress from doing nothing to a policy that achieved 100% success in the reported tests. The robot feedback came from a scripted expert policy and environment rewards.
- 35:07
Limits, risks, and open questions
Cheng notes that Trace can improve code for parallel resource allocation, while an intern also used it to bypass a language model’s safety guardrails in a few iterations. He says current LLM optimizers face context limits, may struggle with unfamiliar project-specific libraries, and do not have established convergence guarantees. Constructive feedback is more useful than a bare success-or-failure signal.
- 37:08
Research directions
Trace currently focuses on workflows that can be represented compactly in text, though its infrastructure could support multimodal inputs. Cheng calls for better optimizers that can handle larger graphs, use memory or multiple agents, and combine language models with search or planning. He also describes a tutorial that uses Trace to tune machine-learning pipeline hyperparameters from training and validation loss.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.