Qwen3-ASR-1.7B DFlash2 draft (pilot)
A DFlash2 draft for greedy speculative decoding with Qwen3-ASR-1.7B (Qwen/Qwen3-ASR-1.7B and Qwen/Qwen3-ASR-1.7B-hf), in the speculators dflash2 format.
- Start: warm-started from the published DFlash draft
wilsonzheng0327/Qwen3-ASR-1.7B-dflashatd62bd4051eand trained further on the same sources. - Offline: in simulated greedy decoding over held-out audio, it accepts 5.0% more tokens per round than that draft on short clips and 6.1% more on long recordings. A DFlash control trained the same way from the same start gains nothing.
- Served: the accept gain did not become a speedup. Measured with sglang-omni on an H200, it runs within noise of the published draft (see Serving below), because each round costs more.
This is a pilot.
- Loading: sglang-omni does not load DFlash2 drafts yet. The served numbers below come from a local, unpublished change that hands this checkpoint to SGLang 0.5.23's DFlash2 draft model and worker.
- Startup: the first start on a machine JIT-compiles a FlashInfer top-k kernel. That took about 13 minutes on an H200 host and is cached afterwards.
| setting | value |
|---|---|
| backbone | the DFlash draft's: fc, hidden_norm, 5 Qwen3 layers (hidden 2048, intermediate 6144, 16 heads / 8 KV heads, head_dim 128), norm |
| DFlash2 additions | grouped dynamic causal convolutions (kernel 2, group 16, identity-initialized) and a candidate selector (rank 256) that reranks the target head's top 16 per position |
| vocabulary | the full 151,936-token vocabulary; the draft has no lm_head of its own and uses the target's |
| target aux layers | 2, 14, 25 (hidden states entering those layers) |
| block size | 8 (7 proposals per round, anchor = last committed token), causal within the block |
| mask token | 151705 |
| sliding window | 32,768 |
| parameters | 353.2M (backbone 264.3M, selector 78.3M, convolutions 10.6M) |
| precision | bf16 (cast from the float32 training checkpoint) |
Training
- Start: the published DFlash draft's backbone. The convolutions start as the identity and the selector from scratch.
- Data: the published draft's sources and long spans. Those are LibriSpeech train-clean-360, People's Speech (clean), VoxPopuli en, Emilia-YODAS en/zh/ja (CC BY; the CC BY-NC Emilia part is not used), AISHELL-1, Common Voice 17 ja, and FLEURS en/zh/ja train.
- Short clips: a random 180,000 of the 591,786 short clips were relabelled, and 150,000 of them were used.
- Long spans: all 23,936 long spans (30-240 s) were used.
- That is 173,936 utterances and 16.8M label tokens.
- Labels: the target's own greedy transcripts (vLLM, empty system context, language auto), as for the published draft.
- Recipe: as the published draft's phase 2. Cross-entropy with D-Pace position weights, plus the selector loss (weight 1.0, misses replaced by the label). AdamW lr 3e-4, 3% warmup, window 32,768, packed 16k-token batches, 3,072 anchors per batch.
- Run: one H200 with speculators at
7ed657f, 30,500 steps planned (5.58 epochs). The shipped checkpoint is step 16,181, which had the best short + long accept on the 360-clip check subset; accept was flat after it.
Accept length
These numbers come from simulated greedy speculative decoding on CUDA with a bf16 target. The sets and labels are the published draft's held-out sets and the same target labels. The published draft's own row reproduces its card (5.43 short).
- One round: from each anchor the draft proposes 7 tokens. The longest prefix matching the target's greedy transcript is accepted, and the target adds one token. Accept = generated tokens / rounds.
- With the selector: the 7 tokens come from a greedy walk over the top-16 candidates. Slot 1 is conditioned on the anchor, and each later slot on the token chosen before it.
- Without the selector: each slot takes the target head's argmax over the draft's hidden state.
| draft | Libri clean | Libri other | FLEURS en | Emilia en | FLEURS zh | Emilia zh | CV ja | FLEURS ja | Emilia ja | en | zh | ja | all | 60 s en | 60 s zh | 60 s ja | 180 s en | 180 s zh | 180 s ja | long all |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| published DFlash draft | 6.43 | 5.88 | 5.79 | 6.12 | 5.62 | 5.59 | 4.51 | 4.99 | 4.32 | 6.09 | 5.60 | 4.61 | 5.43 | 6.24 | 5.74 | 4.74 | 6.15 | 5.64 | 4.69 | 5.57 |
| DFlash control | 6.43 | 6.04 | 5.87 | 6.08 | 5.64 | 5.57 | 4.52 | 5.00 | 4.28 | 6.13 | 5.60 | 4.60 | 5.44 | 6.23 | 5.86 | 4.73 | 6.21 | 5.77 | 4.65 | 5.60 |
| this draft, with the selector | 6.56 | 6.13 | 6.07 | 6.37 | 5.96 | 5.76 | 4.86 | 5.34 | 4.59 | 6.32 | 5.84 | 4.93 | 5.70 | 6.53 | 6.14 | 5.09 | 6.47 | 6.01 | 5.00 | 5.91 |
| this draft, without the selector | 6.46 | 6.05 | 5.98 | 6.26 | 5.89 | 5.62 | 4.56 | 5.04 | 4.30 | 6.22 | 5.73 | 4.63 | 5.51 | 6.31 | 5.86 | 4.75 | 6.35 | 5.76 | 4.63 | 5.64 |
- The control: it is the published draft trained with the same data, recipe and steps on a second H200, with no DFlash2 additions and its 48,000-token head. It separates the gain from DFlash2 from the gain from training longer. Its weights are not published.
- Where the gain comes from: almost all of it is the selector.
- Per-set files: per-set results, including slot-1 accuracy, are in
eval/.
Serving
Setup:
- Stack: sglang-omni with SGLang 0.5.23 on one H200. Both drafts propose 7 tokens per round.
- Workloads: SeedTTS-eval English and Chinese clips, 400 unique clips each, greedy, concurrency 1, 4 and 16.
- Repeats: each draft ran two passes, with the order of its draft and no-draft legs swapped between them.
- Speedup: draft throughput (mean of both passes) over no-draft throughput (mean of all six no-draft legs).
| draft | served accept en | served accept zh | en c1 | en c4 | en c16 | zh c1 | zh c4 | zh c16 |
|---|---|---|---|---|---|---|---|---|
| published DFlash draft | 4.57 | 3.97 | 1.74x | 1.56x | 1.28x | 1.53x | 1.36x | 1.13x |
| this draft | 4.70 | 4.25 | 1.70x | 1.52x | 1.26x | 1.55x | 1.39x | 1.17x |
- What the table shows: served accept is 3% higher in English and 7% higher in Chinese, but throughput is within noise of the published draft. The same draft differed by up to about 5% between passes.
- Why: each round costs about 5% more, which cancels the gain. That figure is estimated from accept and throughput, not profiled. The full-vocabulary head and the selector's top-k and walk are the likely costs.
- Accuracy: WER (en 1.44-1.48%) and CER (zh 0.38%) were unchanged for every draft.
- Regressions: no draft was slower than no draft at any concurrency.
License
Apache-2.0, as the target model and the published DFlash draft.