LFM3-1.7B-DSpark

A lossless DSpark draft model for LiquidAI/LFM3-1.7B, trained with speculators (training code).

Up to 3.14× faster on math and 2.63× on code, 2.88× on average (default generation params)
Mean acceptance length 4.10 on math and 3.24 on code, 2.83 overall (block size 8)

Usage

vLLM (recommended)

Requires vLLM from PR #66323 (Python-only changes, so precompiled kernels can be reused):

VLLM_USE_PRECOMPILED=1 uv pip install \
  "vllm @ git+https://github.com/vllm-project/vllm.git@refs/pull/66323/head"

Then serve:

vllm serve LiquidAI/LFM3-1.7B --port 8000 \
  --speculative-config '{"model": "rasyosef/LFM3-1.7B-DSpark", "num_speculative_tokens": 8}'

llama.cpp (GGUF)

A GGUF build of this drafter is available at rasyosef/LFM3-1.7B-DSpark-GGUF. Pair it with a LFM3-1.7B GGUF target:

llama-server -hf LiquidAI/LFM3-1.7B-GGUF:Q8_0 \
  -hfd rasyosef/LFM3-1.7B-DSpark-GGUF:Q8_0 \
  --spec-type draft-dspark --spec-draft-n-max 4 --spec-draft-n-min 0 \
  -fa on -ngl 99

Details

~0.3B params, bfloat16, block size 8
Draft vocab 32,000; aux hidden states from verifier layers 3/11/19/23
100,000 Open PerfectBlend prompts (filtered) with verifier-generated responses, sample length 2560, packed to 8192
4 epochs, lr 4e-4, cosine (4% warmup), 98/2 train/val split, loss {"ce": 0.1, "tv": 0.9}

Evaluation

RedHatAI/speculator_benchmarks, default generation params (temperature 0.1, top_k 50, repetition_penalty 1.1 from the model's generation_config.json), single A100 40GB, concurrency 1.

Throughput (tokens/s)

Mean output throughput. Baseline is LFM3-1.7B without speculative decoding.

subset	baseline (no drafter)	DSpark (this model)	speedup
math_reasoning	220.8	692.9	3.14×
HumanEval	221.0	580.2	2.63×
average	220.9	636.6	2.88×

Acceptance at default params

acceptance_length is mean tokens committed per verification step, including the bonus token (max 9.0 at block size 8).

subset	acceptance_length	pos_0	pos_1	pos_2	pos_3	pos_4	pos_5	pos_6	pos_7
math_reasoning	4.101	78.5%	61.1%	47.7%	37.4%	29.3%	23.3%	18.4%	14.5%
HumanEval	3.243	71.4%	49.1%	34.4%	24.3%	17.4%	12.4%	8.9%	6.3%
rag	2.971	68.7%	46.4%	31.1%	20.7%	13.4%	8.4%	5.2%	3.1%
tool_call	2.684	64.4%	39.7%	24.9%	15.8%	10.2%	6.5%	4.1%	2.7%
translation	2.630	62.6%	39.4%	24.4%	15.1%	9.4%	6.0%	3.7%	2.3%
writing	2.586	61.4%	36.7%	22.7%	14.7%	9.7%	6.3%	4.2%	2.9%
question	2.573	61.6%	36.7%	22.4%	14.2%	9.4%	6.1%	4.2%	2.8%
summarization	2.467	63.1%	37.4%	21.4%	12.4%	6.7%	3.4%	1.6%	0.7%
qa	2.446	61.6%	36.2%	21.0%	12.0%	6.8%	3.8%	2.1%	1.2%

Weighted across all subsets: 2.832 over 485,518 verification steps.

Limitations

Only works with LFM3-1.7B. Throughput was measured only on math_reasoning and HumanEval. Temperature 0, concurrent load, and contexts beyond the 2,560-token training samples were not evaluated.

License

Apache 2.0

rasyosef/LFM3-1.7B-DSpark