Domain-Adapted Drafters

Headline findings

The domain-shift penalty is large and well quantified. A generalist drafter loses roughly a quarter to a third of its acceptance when the workload or the target moves out of its training distribution. SambaNova measured a generic Llama-3.2-1B drafter falling from 60.7% to 37.5% token acceptance on a biology-tuned target and from 49.6% to 35.7% on Chinese (2503.07807). TAPS measured HASS heads at accepted length 5.02 vs 4.09 on GSM8K depending on whether they were trained on MathInstruct or ShareGPT, and 3.98 vs 2.90 on MT-Bench the other way round (2603.27027). HedgeSpec's specialist pool shows a Python drafter at 7.89 mean accepted tokens on Python but 2.86 on biology (2510.20064). The worst case is a fine-tuned target: an EAGLE-style drafter trained for Qwen2.5-7B collapses to τ ≈ 1.2–2.2 on Qwen2.5-Math / -Coder / Meditron3 variants, versus 4.0–4.8 after re-adaptation (EDA, 2603.09527). Even for a single generic head, per-domain accepted length spreads ~1.5× (SPEED-Bench: coding/math ≈ 3.0–3.5 vs roleplay/writing ≈ 1.9–2.5).

One lever dominates: regenerate responses with the exact target, in the served mode. Every 2025–26 production recipe (SpecForge, Red Hat Speculators, NVIDIA ModelOpt/NeMo, Baseten, Nebius, Modal, AMD, Together, AngelSpec) converges on: take prompts from the deployment domain (production logs or a representative open set), have the target itself write the responses greedily in the reasoning mode that will be served, and train on that. SpecForge v0.3 calls this "the single largest lever on final acceptance". Synthetic prompts (Magpie) reach 80–93% of what real user queries give (2503.07807). Domain adaptation is a change of prompt set, not of algorithm.

Offline white-box KD > black-box SFT > online distillation, when a corpus exists (SambaNova: offline beats online by 11–25%, white-box beats black-box by 2–10%). On-policy distillation under real verification (Draft-OPD, 2605.29343) then breaks the offline plateau (+23% over EAGLE-3 on thinking models). Online adaptation is best framed as drift insurance — NVIDIA's NeMo-RL ablation shows it adds almost nothing with a good domain init (1.77× → 1.78×) but recovers a poor init (1.51× → 1.63×).

Serving many domains: pool + router beats one mixed head; weight-merging fails. TAPS shows checkpoint averaging of two domain heads is worse than either (τ ≈ 2.6), while confidence routing (4.8) or merged-tree verification (5.1) recovers the specialist number. MetaSD (UCB bandit), TaskSpec (prompt classifier), HedgeSpec (full-information no-regret) and OnlineSPEC's Ens-Eagle (Hedge ensemble) all instantiate the pool design. Parameter-efficient specialisation (EDA's private expert at 27.5% of params, TaskSpec's per-task LoRA, the vLLM DFlash-LoRA RFC at ~28× smaller, Turbo LoRA's per-adapter speculator) makes a per-domain drafter cheap enough to ship with every fine-tune.

Measured business numbers for workload-specific drafters. Fireworks: 29% → 76% draft hit rate (generic → custom), ~2× latency on Cursor code-gen. Together: +1.23–1.45× decode speed over the base speculator from 10–50M tokens of customer traffic, 23–26% fewer GPU-hours. Baseten: +20% median acceptance from live training. Nebius LK losses: +8–10% accepted length from the loss alone. Counter-example: Thoughtworks found a custom EAGLE-3 for a domain-adapted Nemotron 3.5 Lightning "could only match" the co-trained native MTP head.

Which domains are easy and hard. Code, math and structured output (JSON, SQL, function calling) give the longest accepted spans; open chat, roleplay, creative writing, summarisation, and low-resource languages the shortest. Language is a domain in its own right: per-language drafters give 1.89× vs 1.18× for a generic one (2406.16758), a 13% acceptance gap separates the fastest and slowest MGSM languages (2510.02128), and task-distilled low-resource-language drafters overfit (α 0.60 in-task but 0.43 off-task, 2605.30580). Long context and RL rollouts are also "domains" — EAGLE-3's accepted length drops from ~5.8 to 1.28 on long inputs (OWL, 2510.07535), and RL policy drift stales a fixed drafter within an epoch (ReSpec, TLT).

Papers — offline domain-drafter training and distillation

Papers — online, continual, and RL-drift adaptation

Papers — drafter pools, routing, and parameter-efficient specialisation

Benchmarks and analyses quantifying domain shift

Blogs, repos, and model zoos

Practical recipe distilled from the sources

Start from a well-initialised generalist head (SpecBundle / RedHatAI / NVIDIA release, or train one on Perfect-Blend-scale data, since breadth alone adds ~20% on math). Collect 100–300K prompts from the deployment domain — production logs if available, otherwise Magpie-style synthetic prompts, which recover 80–93% of the real-log benefit. Regenerate every response with the exact target, greedily, in the reasoning mode and at the temperature that will be served. Train offline with white-box KD (target hidden states and logits), keeping only assistant turns in the loss; for a fine-tuned target, retrain only a private expert or LoRA slice (25–30% of parameters) rather than the whole drafter. If a strict acceptance objective is available (LK losses, EAL/WTV) expect a further 4–10%. Then optionally run on-policy distillation under real verification to break the offline plateau. For multi-domain serving keep separate specialists and combine them at inference — Hedge/UCB routing or merged-tree verification — never by weight averaging. For non-stationary traffic (RL rollouts, new products, per-user sessions) add an online path: OSD/TIDE-style asynchronous retraining triggered by acceptance-rate drift, or a training-free suffix-tree/n-gram layer that adapts by construction. For repetitive agentic, SQL or code-edit traffic the training-free layer may beat any trained drafter outright.

Gaps

No verified legal- or finance-specific drafter papers exist; medical appears only via EDA's Meditron experiment. No public benchmark yet has an agentic/tool-calling split (AgentSpec and SambaNova's function-calling numbers are the only data). Explicitly domain-labelled open heads are rare — z-lab's Qwen3-Coder DFlash, LayerSkip Code Llama, AngelSlim's modality heads, and HedgeSpec's 21-drafter pool are the main ones; everything else is a chat mix with a per-domain eval table. Nobody has published a study of drafter adaptation under pipeline-parallel / WAN serving, where a domain drafter's higher acceptance translates directly into fewer verification round-trips; the numbers above (25–40% acceptance recovery, 1.2–1.5× extra decode speed) are the input for that estimate.

Not found / unverified

"SpecActor" (2511.16193) exists but drafter-training details are not in the abstract; "Speculative Decoding Meets Domain Shift" and "Redistributing Draft" do not resolve to papers; Baseten's live-training +20% figure comes from an X post; the vLLM DFlash-LoRA "within 2%" claim is an RFC, not a paper; SambaNova's SCOPE workshop venue is per the authors.