Spark-X2.5-4B — GGUF quants

GGUF conversions of XHToken/Spark-X2.5-4B (Apache-2.0, xAI Spark lineage — parent xai/grok-code-fast-1). All credit for the model goes to XHToken and xAI; this repo only provides the converted/quantized weights.

Spark-X2.5-4B is a compact reasoning model: 4.11B params, hybrid sliding-window attention (1:3 ratio, 4 window layers per 1 global layer, window 128), native 1M-token context.

Files

Path	Quant	Size	Notes
`iq4_xs/Spark-X2.5-4B-IQ4_XS.gguf`	IQ4_XS (4.25 bpw)	2.22 GiB	best general quality-per-byte
`tq3_4s/Spark-X2.5-4B-TQ3_4S.gguf`	TQ3_4S arch-tuned (4.81 bpw)	2.35 GiB	fastest hard-coding variant (Hard86 46.5%) — requires turbo-tan/llama.cpp-tq3
`bf16/Spark-X2.5-4B-BF16.gguf`	BF16	8.22 GiB	reference quality

Required runtime

IQ4_XS / BF16: any recent upstream llama.cpp with spark2_5 support (PR #27868).
TQ3_4S: turbo-tan/llama.cpp-tq3 fork (custom turbo four-scale quant type), branch main ≥ 46b6224a0.

Quick start

llama-server -m Spark-X2.5-4B-IQ4_XS.gguf --jinja -ngl 99 -c 32768 -fa on

--jinja is mandatory — the Spark2.5 chat template is custom; without the flag llama.cpp refuses to load ("custom template is not supported").

Thinking control

The model is a reasoning model and emits <think> blocks by default. With thinking enabled it can consume the whole token budget on reasoning and return empty content for code-style prompts. Disable per-request:

{"messages": [...], "chat_template_kwargs": {"enable_thinking": false}}

Direct-answer and code-eval workloads should run with thinking off (or a very large max_tokens). All scores below were measured with thinking disabled.

Benchmarks

RTX 3090, temp 0, official scorers (EvalPlus, Hard86 test harness), r=3 for speed.

Speed (llama-bench, ngl=99)

Quant	pp512 (t/s)	tg128 (t/s)
BF16	6724	93.3
IQ4_XS	9102	213.6
TQ3_4S (arch-tuned)	7736	215.2

Quality (thinking off)

Suite	BF16	IQ4_XS	TQ3_4S
Hard86 (86 tests)	54.7%	44.2%	46.5%
HumanEval	78.0%	73.8%	71.3%
HumanEval+	73.8%	70.1%	67.7%
MBPP	72.5%	69.6%	68.8%
MBPP+	62.2%	61.4%	59.3%
Internal quality suite (overall)	75.7	74.1	73.9

How to choose: IQ4_XS for general use (best EvalPlus numbers, smallest 4-bit file). TQ3_4S for hard coding tasks at the same decode speed (+2.3pp Hard86, −2.5pp HumanEval, fork runtime required). BF16 when you need reference quality and can pay 2.3× in speed.

TQ3_4S arch-tuning note

The stock uniform TQ3_4S recipe performs poorly on this architecture (HumanEval 43.3%). The shipped file uses an arch-tuned tensor recipe — fused attn_qkv and ffn_down at Q5_K, attn_output at Q4_K, attn_gate at F16, rest turbo four-scale — developed and validated in this campaign (HumanEval 43.3 → 71.3, Hard86 17.4 → 46.5). attn_output specifically must stay Q4_K: Q5_K there collapses decode speed (215 → 86 t/s) by breaking the fused kernel path.

Conversion recipe

# IQ4_XS
python convert_hf_to_gguf.py <source-dir> --outtype bf16 --outfile Spark-X2.5-4B-BF16.gguf
llama-quantize Spark-X2.5-4B-BF16.gguf Spark-X2.5-4B-IQ4_XS.gguf IQ4_XS

# TQ3_4S (arch-tuned, turbo-tan/llama.cpp-tq3)
llama-quantize --tensor-type "attn_qkv=q5_k" --tensor-type "attn_gate=f16" \
  --tensor-type "ffn_down=q5_k" --tensor-type "attn_output=q4_k" \
  Spark-X2.5-4B-BF16.gguf Spark-X2.5-4B-TQ3_4S.gguf TQ3_4S

License

Apache-2.0, inherited from XHToken/Spark-X2.5-4B.

YTan2000/Spark-X2.5-4B-GGUF