RoboBoom
Matroid· February 2, 2026· 29:32

Ashok Elluswamy: Building Foundational Models for Robotics at Tesla

Ashok Elluswamy, who leads Tesla’s AI team, discusses end-to-end driving models, evaluation, and shared AI foundations for Tesla vehicles and Optimus at a Matroid event.

In this talk

  1. 0:00

    Tesla’s vision for autonomy and robotics

    Elluswamy frames Tesla’s mission as making abundance more widely available through autonomous vehicles and humanoid robots. He describes the Austin robotaxi service and presents Optimus as a way to automate physical work using spaces designed for people.

  2. 3:30

    Why Tesla uses end-to-end driving models

    Tesla’s driving network takes camera video, navigation, vehicle state, and audio, then predicts control actions. Elluswamy argues that separate perception, prediction, and planning modules can lose context, using a puddle and oncoming traffic to show why driving decisions depend on the whole scene. He also cites FSD examples where cars wait for chickens or move around geese based on their behavior.

  3. 8:01

    Fleet data and predictive safety

    Elluswamy describes the challenge of turning large streams of camera history into the right driving actions, and says Tesla’s fleet generates about 500 years of driving data daily. The team filters for unusual events, such as emergency vehicles or a car beginning to skid, so models can learn from rare situations and react early.

  4. 13:30

    Reasoning and simulation for evaluation

    Tesla probes its end-to-end network with geometric and language-based reasoning, including 3D scene representations generated with Gaussian splatting. For evaluation, a learned world simulator recreates video from actions, allowing newer policies to replay past incidents or face modified scenarios. Elluswamy says the same modeling approach extends to Optimus scenes and manipulation tasks.

  5. 21:49

    Shared models and audience questions

    Elluswamy says Tesla trains driving and simulation networks on data shared across its robots, showing controllable indoor scenes and actions such as opening drawers. In the Q&A, he argues cameras provide enough information when AI can interpret them, says the system issues control commands 36 times per second, and discusses future voice control alongside safety concerns.

Briefs summarise talks and interviews; timestamps open the video at that moment. Claims in a talk are the speaker's own.

More talks