NCM Trajectory

Watch a Next-Concept Model write a story in parallel sweeps, coarse to fine.

What you'll learn: NCM (HazyResearch) doesn't write token by token. It generates a hierarchy of concept codes coarse to fine: one forward pass per level. Replay real generations below and you'll see the same story sharpen 12 times: after half the sweeps it is word salad with the characters already in place; it only becomes readable in the last three.
What to do: pick a story, press Play, or click any level to jump.
story:

What is going on here

A Next-Concept Model compresses text with a multiscale residual VQ tokenizer: the same 128-position story becomes 12 nested sets of discrete codes, from 1 code (the topic) to 128 codes (every position). The generator predicts that hierarchy coarse to fine; positions inside a level come out in parallel.

The texts above are not samples of increasing quality: they are the tokenizer's partial reconstruction of the codes generated so far, after each of the 12 sweeps of one real generation. Green words are new at this level; struck-through words disappeared.

Reproduce it

Trajectories were captured on CPU with the released TinyStories checkpoint using tools/capture_trajectories.py in this repo (it reuses the model code of HazyResearch/ncm, MIT). Run it on any NCM checkpoint or prompt set and drop the JSON here.

Paper: Next-Concept Modeling (arXiv 2610.14162) · Code + data: iamdgarcia/ncm-trajectory · Built by iamdgarcia.