Spark-X2.5-4B GGUF

Community GGUF quantizations of XHToken/Spark-X2.5-4B.

☕ If this GGUF made your day easier, a coffee would make mine.
Send a coffee ☕
I build and test these releases myself. Your coffee helps keep me going.
Thank you for supporting this work.

About Spark-X2.5-4B

Spark-X2.5-4B is a compact, general-purpose language model developed by SparkLLM. According to the upstream authors, the model is designed for:

• General-purpose capabilities: conversation, writing, translation, reasoning, and coding
• Tool use and agentic workflows
• Native context window of up to 1M tokens
• More than 200 supported languages
• Efficient hybrid attention: one full-attention layer combined with three sliding-window attention layers
• Broad ecosystem support: llama.cpp, vLLM, SGLang, MLX, Ollama, and LM Studio

The upstream model is also designed for broad hardware compatibility and efficient long-context inference.

For the original model architecture, training details, benchmarks, and official usage instructions, see the official model card.

This repository is a quantization-only release for local inference. No model training or fine-tuning was performed.

Recommended quantized profiles

These are three practical starting points, all excluding the BF16 reference:

Fidelity measurements

The table below compares every published GGUF file with the BF16 reference on a held-out WikiText-2 evaluation: eight chunks from wiki.test.raw and eight chunks from wiki.valid.raw, using a 4,096-token context and the same Spark llama.cpp runtime. Values are averaged across the two splits. The BF16 reference mean PPL was 20.718057 in this evaluation. Lower Mean KLD, ΔPPL, and RMS Δp, and higher Top-1 agreement, indicate closer next-token behavior to BF16. These are next-token fidelity measurements, not a direct percentage of capabilities retained and not a substitute for task-specific evaluation.

The machine-readable results are available in reproducibility/quality-summary.tsv, with corpus hashes, evaluation settings, calibration provenance, and runtime details in reproducibility/manifest.md.

Quick start

./llama-cli \
  -m Spark-X2.5-4B-Q4_K_M.gguf \
  --chat-template-file chat_template.jinja \
  --jinja \
  -p 'Answer briefly in English: What is GGUF and why is it useful for running language models locally?' \
  -n 128 -c 4096 -ngl 99

Reproducibility and validation

This is a quantization-only release: no training or fine-tuning was performed. The BF16 source, runtime revision, clean calibration split, importance matrix, quantization commands, checksums, and validation profile are recorded in the reproducibility/ package. All 16 published files passed load/generate validation on an NVIDIA A10M.

License and attribution

The upstream model is released under Apache License 2.0. Preserve the upstream attribution and license when redistributing these derivative files. This is a community GGUF quantization, not an official XHToken/SparkLLM release or endorsement.

Checksums are available in SHA256SUMS.txt.