DSpark

Speculative decoding speeds up a large model by having a small, cheap drafter guess the next several tokens, then letting the big model check them all in one pass and keep the good ones. The output is identical; you just wait less. The whole trick lives or dies on the drafter, and the field has been stuck choosing between two flawed ones: guess the tokens one at a time (accurate, but slow), or guess a whole block in parallel (fast, but the guesses decay toward the end of the block because each token was drafted blind to the ones before it).

Pure parallel drafting decays on the suffix, so you bolt a sequential corrector onto it — the exact opposite bet from the diffusion drafters that sold parallelism as the whole answer.

DSpark, which DeepSeek has now put into production on its V4 models, refuses the choice. It drafts the block in parallel for speed, then runs a small second head back over that block to re-inject the dependency the parallel pass threw away — a corrector that asks, token by token, 'given what came before, is this still the right guess?' A third head reports how confident the draft is, which lets the server spend more verification effort on requests likely to pay off and less on the rest, scaling to how busy the GPUs are at that moment.

The payoff is the part that matters: this replaced DeepSeek's previous drafter in live serving, not in a benchmark. Under real traffic it generates tokens 57 to 85% faster for each user, and — because the same GPUs now clear work faster — pushes 51% more total traffic through the fleet at DeepSeek's normal speed target. For an inference provider, that 51% is a straight cut in the machines needed to serve the same demand. The company also open-sourced DeepSpec, an MIT-licensed toolkit for training your own drafter on open models like Qwen and Gemma, where its guesses run longer before the big model rejects one than the best prior open methods.

One number to distrust: some coverage repeats a 661% throughput figure. The paper's own footnote defuses it — that ratio only appears at a punishing speed target where the old drafter had all but collapsed, so it measures a near-zero baseline, not a 6.6x steady-state win. The steady win is the 51%, and it is already spreading: vLLM, the serving stack much of the open-model world runs on, has folded DSpark in natively.