Ep#66: Ordered Action Tokenization

In this talk

1. 5:02

   Why FAST tokens can fail to decode

   FAST applies byte-pair encoding to flattened Fourier coefficients, so its tokens can represent a variable number of coefficients. The decoded sequence may not fill the fixed-size frequency matrix. Dropping excess coefficients or padding missing ones can shift action dimensions or introduce invalid values, so Chi considers those fixes unsafe.

2. 8:55

   Ordering tokens for useful prefixes

   Chi argues that action representations should be easy for generative models to learn, alongside balancing compression and reconstruction fidelity. OAT uses learnable registers and nested dropout: training randomly removes tokens from the end, requiring the decoder to reconstruct from each remaining prefix. Early tokens therefore capture common action patterns, while later tokens add detail.

3. 12:09

   Variable detail at inference

   An autoregressive policy can generate one token for a coarse action chunk, then continue generating tokens for finer reconstruction. In the KUKA trajectory example, one token captures the general motion direction, while four tokens nearly reproduce the trajectory. Chi says four tokens can outperform eight on one task because extra generation steps may compound errors.

4. 17:02

   Choosing token budgets and measuring speed

   Chi describes using fewer tokens for simple motion and more for precise, contact-rich actions, such as shoe tying. Estimating the needed token count online remains future work; one possible approach is to predict a token budget from the observation. With a small policy and KV caching, one-token inference was at least twice as fast as eight-token inference, though observation encoding took much of the runtime.

5. 32:51

   Where action tokenization could go next

   Chi traces action encoders from flattened MLP inputs to temporal convolutions and learnable registers, noting that nested dropout fits encoders whose registers can access the full action chunk. He also discusses discrete action tokens as auxiliary training targets alongside continuous flow or diffusion policies. Future questions include balancing reconstruction fidelity with token overlap, conditioning on embodiment, and using token representations in pretraining or post-training.