LiLiCorr speculator for Qwen3.5-4B
A LiLiCorr draft model (arXiv:2608.20530) for Qwen/Qwen3.5-4B:
a DFlash backbone plus a candidate-lattice head that rescores the backbone's per-slot
top-k candidates, so serving can commit a whole greedy path through the lattice
instead of only top-1 tokens.
Trained with speculators.
Validation results
Held-out 10% split of the training corpus, end of epoch 1.
| metric | value |
|---|---|
lilicorr_eal (lattice) |
7.100 |
eal |
6.153 |
lilicorr_top1_eal |
6.157 |
lilicorr_oracle_eal (headroom) |
9.579 |
| full-block accuracy | 0.4569 |
The lattice head is doing real work: lilicorr_eal (7.100) runs ahead of
top1_eal (6.157), with oracle headroom at 9.579.
Per-position acceptance
| position | accuracy |
|---|---|
| 1 | 0.8846 |
| 2 | 0.7812 |
| 3 | 0.6972 |
| 4 | 0.6258 |
| 5 | 0.5629 |
| 6 | 0.5065 |
| 7 | 0.4559 |
| 8 | 0.4099 |
| 9 | 0.3689 |
| 10 | 0.3323 |
| 11 | 0.2987 |
| 12 | 0.2688 |
| 13 | 0.2421 |
| 14 | 0.2180 |
| 15 | 0.1963 |
Training data
100,401 conversations — a deterministic, hash-selected subset of open-perfectblend regenerated on-policy by Qwen3.5-4B itself (1,729,783 conversations available in the full corpus). ~443.6M tokens, 89.9% supervised.
Note: this is a ~5.8% subset, not the full regenerated corpus. The on-policy regeneration — responses generated by the same model being drafted for — is the main reason acceptance is high relative to drafters trained on data regenerated by a different model.
Training configuration
| epochs | 1 (40,091 global steps) |
| optimizer | muon, lr 6e-4, weight decay 0 |
| schedule | cosine, 4% warmup |
| sequence length | 3072 |
| max anchors | 512 |
| block size | 16 (15 speculative tokens) |
| draft layers | 5, from verifier hidden layers 2/9/16/23/30 |
| loss | CE, fixed-exp-decay per-position weighting (gamma 7), ttt_steps 3 |
| parallelism | 2-way data parallel |
| seed | 42 |
Built with speculators 0.10.0a13, vLLM 0.32.0, transformers 5.17.0, torch 2.13.0+cu130.
Usage
Serve with vLLM alongside the verifier. The shape of the speculative config used in internal testing:
vllm serve Qwen/Qwen3.5-4B \
--speculative-config '{"method": "dflash", "model": "MeganEFlynn/Qwen3.5-4B-speculator.lilicorr", "num_speculative_tokens": 15, "draft_sample_method": "greedy"}'
See the speculators documentation for details.
Limitations
- Trained on a 5.8% subset for a single epoch; not a fully-converged release artifact.
- Validation metrics are on a held-out split of the training distribution, not an independent serving benchmark. No SpeedBench or acceptance-under-serving numbers are published here.
- Draft sampling is greedy; the full verifier vocabulary is retained.