RoboBoom
The TWIML AI Podcast with Sam Charrington· September 1, 2026· 1:05:31

Are World Models What Comes After Language Models?

Justin Johnson, co-founder of World Labs and associate professor at the University of Michigan, discusses world models with Sam Charrington on The TWIML AI Podcast.

In this talk

  1. 3:19

    What counts as a world model

    Johnson says researchers use “world model” for several distinct ideas: implicit knowledge of the real world, models that predict state changes in reinforcement learning, and generative systems that create navigable environments. He favors clearer terminology because these are all valuable research directions, but they are not the same kind of system. He also distinguishes generating realistic observations from building theories that explain how the world works.

  2. 11:37

    Agents, states, and actions

    Johnson explains the POMDP framework as an agent acting on a world, receiving partial observations, and responding as the world changes. The state is an abstraction suited to the task, and a learned internal representation can behave like a state even when the true physical state is inaccessible. A robot trained by behavior cloning can operate in this loop without being trained with a reinforcement learning reward.

  3. 24:14

    Explicit 3D and generated pixels

    Johnson contrasts explicit 3D representations such as Gaussian splats with models that generate pixels directly. World Labs’ Marble turns text, images, or video into splat-based worlds, while its RTFM research model generates interactive frames without an explicit 3D representation. Marble first generates a 360-degree panorama, then lifts it into 3D; a single image can anchor the visible scene while the model fills in unseen areas.

  4. 47:46

    A taxonomy and a path to unified models

    Johnson groups world models by what they produce in the agent-world loop: renderers output observations, planners output actions, and simulators model how states change. Marble spans rendering and simulation because its splats can be viewed and manipulated as a 3D state. He expects these capabilities to increasingly share a model, with different operating modes for tasks such as controlling a robot, generating a virtual world, or exploring counterfactuals.

  5. 59:29

    Scaling models to spatial contexts

    Johnson expects transformers to remain useful, while training objectives and methods for handling long contexts continue to evolve. World models may need hundreds of thousands or millions of tokens to represent video or 3D space. Tokens could correspond to video patches, groups of splats, spatial chunks, or abstract latent states; he advises choosing representations that fit the task and remain simple enough to scale.

Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.

More talks