Jason Ma, researcher at the University of Pennsylvania, presents research on safer and reward-free offline reinforcement learning at a GRASP Lab seminar.
Ma explains that offline reinforcement learning trains a policy from a fixed dataset, without new interaction with the environment. This can suit settings such as healthcare and autonomous driving, where collecting live data is costly or dangerous. He argues that average return can hide risk: three slot-machine options may each average zero while having very different downside, so a risk-averse decision maker would favor the least volatile.
Ma asks whether a safer policy can be learned from data collected without regard to risk. His CODAC method applies conservative penalties to return quantiles, with stronger penalties for actions less represented in the dataset, helping limit errors from distribution shift. In a navigation experiment with a hazardous region, CODAC’s CVaR variant learned to avoid that region from risk-neutral data and performed better on the risk-sensitive measure than the compared methods.
Ma turns to tasks where writing a reward function is difficult, and proposes using demonstrations or examples of successful outcomes instead. His approach matches distributions over task-relevant states, so expert observations need not include actions or come from the same type of agent. This also lets the method use cheaper, suboptimal data alongside a smaller expert dataset.
Ma introduces SMODICE, which converts state-distribution matching into three separate supervised learning steps using offline data. The method estimates weights for dataset samples and uses weighted behavior cloning to train a policy, avoiding the need to sample from the learner’s environment during optimization. Across reported D4RL and manipulation tasks, it was competitive with leading methods, declined less under embodiment mismatch, and could learn tasks from examples of success.
Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.
Jason Ma, Dyna Robotics co-founder and chief scientist, joins Latent Space on site to discuss Dyna-2, robot learning, hardware, and commercial deployments.
Jason Ma, co-founder and AI research lead at Dyna Robotics, presents Dyna 2’s approach to robot learning and commercial deployment at the Fellows Forum on Day 2.
Jason Ma, cofounder of Dyna Robotics, discusses building reliable robot foundation models for real-world manipulation at the Humanoids Summit.
Jason Ma, a robotics researcher at the University of Pennsylvania who worked on the paper at Google DeepMind, joins RoboPapers hosts Michael and Chris to discuss using vision-language models to assess robot task progress and data quality.
Jensen Huang, CEO of Nvidia, describes how AI is being used inside the company and predicts that robotics will reshape factories; AI Explained host discusses these claims with robotics researcher Jason Ma and IPPR report author Carsten Young.
