Spark-X2.5-4B — GGUF quants
GGUF conversions of XHToken/Spark-X2.5-4B (Apache-2.0, xAI Spark lineage — parent xai/grok-code-fast-1). All credit for the model goes to XHToken and xAI; this repo only provides the converted/quantized weights.
Spark-X2.5-4B is a compact reasoning model: 4.11B params, hybrid sliding-window attention (1:3 ratio, 4 window layers per 1 global layer, window 128), native 1M-token context.
Files
| Path | Quant | Size | Notes |
|---|---|---|---|
iq4_xs/Spark-X2.5-4B-IQ4_XS.gguf |
IQ4_XS (4.25 bpw) | 2.22 GiB | best general quality-per-byte |
tq3_4s/Spark-X2.5-4B-TQ3_4S.gguf |
TQ3_4S arch-tuned (4.81 bpw) | 2.35 GiB | fastest hard-coding variant (Hard86 46.5%) — requires turbo-tan/llama.cpp-tq3 |
bf16/Spark-X2.5-4B-BF16.gguf |
BF16 | 8.22 GiB | reference quality |
Required runtime
- IQ4_XS / BF16: any recent upstream llama.cpp with spark2_5 support (PR #27868).
- TQ3_4S: turbo-tan/llama.cpp-tq3 fork (custom turbo four-scale quant type), branch
main≥46b6224a0.
Quick start
llama-server -m Spark-X2.5-4B-IQ4_XS.gguf --jinja -ngl 99 -c 32768 -fa on
--jinja is mandatory — the Spark2.5 chat template is custom; without the flag llama.cpp refuses to load ("custom template is not supported").
Thinking control
The model is a reasoning model and emits <think> blocks by default. With thinking enabled it can consume the whole token budget on reasoning and return empty content for code-style prompts. Disable per-request:
{"messages": [...], "chat_template_kwargs": {"enable_thinking": false}}
Direct-answer and code-eval workloads should run with thinking off (or a very large max_tokens). All scores below were measured with thinking disabled.
Benchmarks
RTX 3090, temp 0, official scorers (EvalPlus, Hard86 test harness), r=3 for speed.
Speed (llama-bench, ngl=99)
| Quant | pp512 (t/s) | tg128 (t/s) |
|---|---|---|
| BF16 | 6724 | 93.3 |
| IQ4_XS | 9102 | 213.6 |
| TQ3_4S (arch-tuned) | 7736 | 215.2 |
Quality (thinking off)
| Suite | BF16 | IQ4_XS | TQ3_4S |
|---|---|---|---|
| Hard86 (86 tests) | 54.7% | 44.2% | 46.5% |
| HumanEval | 78.0% | 73.8% | 71.3% |
| HumanEval+ | 73.8% | 70.1% | 67.7% |
| MBPP | 72.5% | 69.6% | 68.8% |
| MBPP+ | 62.2% | 61.4% | 59.3% |
| Internal quality suite (overall) | 75.7 | 74.1 | 73.9 |
How to choose: IQ4_XS for general use (best EvalPlus numbers, smallest 4-bit file). TQ3_4S for hard coding tasks at the same decode speed (+2.3pp Hard86, −2.5pp HumanEval, fork runtime required). BF16 when you need reference quality and can pay 2.3× in speed.
TQ3_4S arch-tuning note
The stock uniform TQ3_4S recipe performs poorly on this architecture (HumanEval 43.3%). The shipped file uses an arch-tuned tensor recipe — fused attn_qkv and ffn_down at Q5_K, attn_output at Q4_K, attn_gate at F16, rest turbo four-scale — developed and validated in this campaign (HumanEval 43.3 → 71.3, Hard86 17.4 → 46.5). attn_output specifically must stay Q4_K: Q5_K there collapses decode speed (215 → 86 t/s) by breaking the fused kernel path.
Conversion recipe
# IQ4_XS
python convert_hf_to_gguf.py <source-dir> --outtype bf16 --outfile Spark-X2.5-4B-BF16.gguf
llama-quantize Spark-X2.5-4B-BF16.gguf Spark-X2.5-4B-IQ4_XS.gguf IQ4_XS
# TQ3_4S (arch-tuned, turbo-tan/llama.cpp-tq3)
llama-quantize --tensor-type "attn_qkv=q5_k" --tensor-type "attn_gate=f16" \
--tensor-type "ffn_down=q5_k" --tensor-type "attn_output=q4_k" \
Spark-X2.5-4B-BF16.gguf Spark-X2.5-4B-TQ3_4S.gguf TQ3_4S
License
Apache-2.0, inherited from XHToken/Spark-X2.5-4B.