Literature review · Pluralis · compiled August–October 2026
When a model's pipeline stages live on different nodes across the internet, every decoded token pays a full lap of link RTTs — and speculative decoding is the one technique that divides the lap count: draft several future tokens cheaply, verify them all in a single parallel pass, provably losing nothing. This review maps that technique for the WAN pipeline-parallel setting: which methods exist, which survive real links, and what to build.
A summary brief plus ten parts: the published method landscape, speculation meeting pipeline parallelism over WAN, two quantitative deep dives — the payload levers that decide WAN throughput, and the scheduling levers that reclaim idle time — a practical drafter catalog for GLM-5.2 and DeepSeek-V4-Pro, the benchmarks that measure it all, our own measured acceptance data for DeepSeek-V4-Pro with its shipped DSpark drafter, a tensor-level parameter ledger of that drafter, a survey of domain-adapted drafters — how much acceptance is lost out of domain and how to train it back — and a survey of tree-based speculative decoding: how draft trees are built, verified and bounded, and why they win at batch 1 but collapse under load.
Summary
The capstone, on one page: where the field stands, the seventeen works that matter most for speculative decoding in WAN pipeline-parallel inference, and twelve open gaps — theory, protocol, compression, and systems — each sized and pointed at the papers it starts from, ending with the stack to build.
Part I
Maps ~45 published methods into five families by where the draft tokens come from, compares their claimed gains, and tracks which are open source and which ship natively in vLLM and SGLang.
Part II
Pipeline-parallel decoding pays two structural taxes: the bubble, and a full traversal of every inter-stage link per token. Which speculative methods are already used with PP, which are attractive but unused, and what a WAN-PP deployment should adopt.
Part III
Deep dive into the two levers that decide WAN throughput: amortize traversals (acceptance-length theory, tree construction, verifier rules) and shrink the wire (exact ~50-byte protocols, 3–4-bit activations, byte accounting) — joined into a bandwidth-aware tree-sizing model.
Part IV
Deep dive into the two scheduling levers: fill the bubbles (in-pipe speculation schedules, the 8–14-stage depth wall, what else can ride idle stages) and overlap async (the max(·) latency law, hedge-branch economics, rollback and cancellation machinery).
Part V
Every worthwhile drafter for GLM-5.2 and DeepSeek-V4-Pro, all speculative methods included: the MTP and DSpark speculators shipped in the checkpoints, third-party heads (DSpark and DFlash now exist for GLM-5.2; still none for V4-Pro), extracted GGUF drafters, training-free options — with serving recipes, measured acceptance lengths, and a vLLM/SGLang/TRT-LLM/llama.cpp matrix. Refreshed September 2026.
Part VI
How SD setups are evaluated: the harnesses (Spec-Bench, SPEED-Bench, SpecDecode-Bench), the datasets per domain — coding, math, reasoning, science, chat, RAG, agentic, long-context, multilingual — the metrics, the measured pitfalls, and a recommended suite. Acceptance is a property of the workload.
Part VII
Our own evaluation of the shipped DSpark drafter on DeepSeek-V4-Pro (K=7, T=1.0, 4.3M verify steps across nine datasets): step-weighted acceptance length by workload — 4.9–5.3 on math/code, 2.8–3.5 on chat — per-position acceptance, step outcome distributions, implied τ at shorter blocks, prompt-difficulty vs sampling noise, and the resulting tok/s ceiling for a 10-stage WAN pipeline.
Part VIII
A tensor-level accounting of the DSpark drafter shipped with DeepSeek-V4-Pro, from safetensors headers alone: 77.50B logical parameters across three MoE MTP blocks (98.2% in 3 × 384 routed experts, ~2.5B activated per step), bit-identical in the DeepSeek and NVIDIA checkpoints — and why NVIDIA's model card says “82B”: it counts 4.76B NVFP4 scale elements as parameters.
Part IX
A survey of 88 papers, blogs, repos and model releases on drafters specialised to a domain, task, language, user, or fine-tuned target: a generic drafter loses 28–38% of its acceptance out of domain and collapses to τ ≈ 1.2 on a fine-tuned target; regenerating domain prompts with the exact target recovers it; specialist pools with a router beat one mixed head; online adaptation is drift insurance. With the offline / online / routing / benchmark tables and a distilled training recipe.
Part X
A survey of 64 tree-based speculative decoding methods, 27 repos and 19 engineering posts (2023 – Oct 2026): static trees (SpecInfer, Medusa, Sequoia) to confidence-shaped dynamic trees (EAGLE-2, OPT-Tree) to cost-aware trees over block drafters (CAST, TreeSpark, CaDDTree). A tree adds ~0.7 accepted tokens/step and 10–20% at batch 1 but drops below 1× at batch 64; accepted length is bounded by O(log P) in verifier capacity. With a filterable method table, engine support matrix and twelve research gaps.
Part XI
What the drafters were actually trained on, with what hardware, and how much of it anyone admits. Three reference tables: training facts for 23 drafter architectures, a disclosure audit of 34 published checkpoints, and the 15 public corpora they draw on. Regenerating responses with the exact target is the largest data lever; draft training fits one 8-GPU node, but the target pass over 1–1.4M prompts and its terabytes of hidden states is the real cost line. Only 4 of 35 releases disclose enough to reproduce the run.