Spark-X2.5-4B GGUF
Community GGUF quantizations of XHToken/Spark-X2.5-4B.
☕ If this GGUF made your day easier, a coffee would make mine.Send a coffee ☕
I build and test these releases myself. Your coffee helps keep me going.
Thank you for supporting this work.
About Spark-X2.5-4B
Spark-X2.5-4B is a compact, general-purpose language model developed by SparkLLM. According to the upstream authors, the model is designed for:
- General-purpose capabilities: conversation, writing, translation, reasoning, and coding
- Tool use and agentic workflows
- Native context window of up to 1M tokens
- More than 200 supported languages
- Efficient hybrid attention: one full-attention layer combined with three sliding-window attention layers
- Broad ecosystem support: llama.cpp, vLLM, SGLang, MLX, Ollama, and LM Studio
The upstream model is also designed for broad hardware compatibility and efficient long-context inference.
For the original model architecture, training details, benchmarks, and official usage instructions, see the official model card.
This repository is a quantization-only release for local inference. No model training or fine-tuning was performed.
Recommended quantized profiles
These are three practical starting points, all excluding the BF16 reference:
| Profile | Choose it when |
|---|---|
Spark-X2.5-4B-Q8_0.gguf |
You want the closest practical profile to the BF16 reference. |
Spark-X2.5-4B-Q4_K_M.gguf |
You want the main size/quality balance. |
Spark-X2.5-4B-Q2_K.gguf |
You need a smaller profile and accept a larger fidelity trade-off. |
Fidelity measurements
The table below compares every published GGUF file with the BF16 reference on
a held-out WikiText-2 evaluation: eight chunks from wiki.test.raw and eight
chunks from wiki.valid.raw, using a 4,096-token context and the same Spark
llama.cpp runtime. Values are averaged across the two splits. The BF16
reference mean PPL was 20.718057 in this evaluation. Lower Mean KLD, ΔPPL, and
RMS Δp, and higher Top-1 agreement, indicate closer next-token behavior to
BF16. These are next-token fidelity measurements, not a direct percentage of
capabilities retained and not a substitute for task-specific evaluation.
| File | Mean KLD ↓ | Top-1 vs BF16 ↑ | ΔPPL | RMS Δp |
|---|---|---|---|---|
Spark-X2.5-4B-Q8_0.gguf |
0.005110 | 97.219% | −0.770% | 1.803% |
Spark-X2.5-4B-Q6_K.gguf |
0.020056 | 93.689% | −2.333% | 3.402% |
Spark-X2.5-4B-Q5_K_M.gguf |
0.057362 | 89.332% | +1.693% | 5.748% |
Spark-X2.5-4B-Q5_K_S.gguf |
0.066414 | 88.541% | −0.175% | 6.175% |
Spark-X2.5-4B-Q4_K_M.gguf |
0.151599 | 83.009% | +4.698% | 8.766% |
Spark-X2.5-4B-Q3_K_M.gguf |
0.303065 | 76.429% | +22.339% | 12.197% |
Spark-X2.5-4B-Q2_K.gguf |
0.904467 | 60.469% | +89.737% | 21.993% |
Spark-X2.5-4B-IQ4_XS.gguf |
0.166476 | 82.248% | +8.917% | 9.248% |
Spark-X2.5-4B-IQ4_NL.gguf |
0.160332 | 82.514% | +7.958% | 9.113% |
Spark-X2.5-4B-IQ3_M.gguf |
0.363476 | 73.510% | +17.084% | 14.386% |
Spark-X2.5-4B-IQ3_S.gguf |
0.424754 | 71.733% | +12.067% | 15.217% |
Spark-X2.5-4B-Q3_K_L.gguf |
0.258238 | 78.081% | +24.321% | 11.409% |
Spark-X2.5-4B-Q2_K_S.gguf |
1.582512 | 51.261% | +202.887% | 29.706% |
Spark-X2.5-4B-IQ2_XS.gguf |
2.931449 | 30.453% | +895.742% | 42.489% |
Spark-X2.5-4B-IQ1_M.gguf |
6.530486 | 7.194% | +31,288.455% | 52.605% |
Spark-X2.5-4B-Q1_0.gguf |
12.363455 | 0.000% | +5,913,223.270% | 55.296% |
The machine-readable results are available in
reproducibility/quality-summary.tsv,
with corpus hashes, evaluation settings, calibration provenance, and runtime
details in reproducibility/manifest.md.
Quick start
./llama-cli \
-m Spark-X2.5-4B-Q4_K_M.gguf \
--chat-template-file chat_template.jinja \
--jinja \
-p 'Answer briefly in English: What is GGUF and why is it useful for running language models locally?' \
-n 128 -c 4096 -ngl 99
Reproducibility and validation
This is a quantization-only release: no training or fine-tuning was performed.
The BF16 source, runtime revision, clean calibration split, importance matrix,
quantization commands, checksums, and validation profile are recorded in the
reproducibility/ package. All 16 published files passed
load/generate validation on an NVIDIA A10M.
License and attribution
The upstream model is released under Apache License 2.0. Preserve the upstream attribution and license when redistributing these derivative files. This is a community GGUF quantization, not an official XHToken/SparkLLM release or endorsement.
Checksums are available in SHA256SUMS.txt.