Feed
Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation
Generate text from a rough sketch to fine detail, filling each level in parallel.
21×
faster
Wall-clock speedup on LM1B
Better generative perplexity than the strongest diffusion model, DUO
How it works
Diffusion is still slow
Diffusion language models need many denoising passes over the full sequence and lose quality when steps are cut.
Coarse-to-fine code levels
A VQ-VAE compresses text into levels of codes, short to long. A transformer predicts each whole level in one pass.
Fewer, cheaper steps
About 100M-parameter models match or beat diffusion baselines 10–31× faster. They also lead on Sudoku with 12 steps.
From the paper