DFlash

Speculative decoding speeds up a large language model by having a small, cheap model guess the next several tokens, then letting the big model check them all in a single pass and keep the good ones. The output is identical to running the big model alone; you just wait less. The bottleneck has quietly become the guesser itself: the best open method, EAGLE-3, still drafts its guesses one token at a time, autoregressively, so a slow drafter caps the whole speedup.

The slow, quality-first way to generate text turns out to be the fast way to guess it.

DFlash, out of UC San Diego's Z Lab, swaps in a different kind of drafter — a block diffusion model, which generates a whole block of candidate tokens in one parallel forward pass rather than left to right. That is the surprise. Diffusion language models are usually sold as the slow, quality-first alternative to ordinary generation; here the one-shot block draft is exactly what makes them the fast option. Draft the block in a single pass, verify it in a single pass, and the guessing stage stops being the drag.

The numbers hold up under independent measurement, which is what raises this above a lab claim. On open models like Qwen and Gemma the drafter runs roughly 2.5x faster end-to-end than EAGLE-3, and up to six times faster than plain token-by-token decoding, with output quality unchanged. NVIDIA, benchmarking on its own hardware, and the SGLang team, testing separately, both reproduced the gains — and DFlash now ships as the default speculative algorithm in SGLang, so anyone already serving open models on that stack inherits the speedup.

The library and the pretrained draft models are up under an MIT licence; the one piece still missing is the training recipe, which the authors say is coming, for anyone who wants to fit a DFlash drafter to their own model. (Modal, whose CTO name-checked the project, is a compute-and-integration collaborator here, not the author — worth stating because the launch has been miscredited to it.)