Spark-X2.6-1.7B GGUF (imatrix quants)
GGUF quantizations of XHToken/Spark-X2.6-1.7B, computed with an imatrix derived from the model's own chat template.
This is an independent, unofficial quantization; it is not affiliated with or endorsed by XHToken. The base model card (architecture, benchmarks, intended use) is at the link above.
TL;DR
- Pick Q4_K_M (1.03 GiB) as the default.
- Q5_K_M (1.17 GiB) or Q6_K (1.31 GiB) if you have the room.
- IQ3_M (848 MiB) if you don't.
- All quants below were made with the shipped
Spark-imatrix.ggufimatrix; seeSpark-imatrix.ggufto requantize with the same calibration.
Files
| file | size | type | notes |
|---|---|---|---|
| Spark-X2.6-1.7B-BF16.gguf | 3.18 GiB | BF16 | reference; not for inference |
| Spark-X2.6-1.7B-Q8_0.gguf | 1.69 GiB | Q8_0 | near-lossless |
| Spark-X2.6-1.7B-Q6_K.gguf | 1.31 GiB | Q6_K | high quality |
| Spark-X2.6-1.7B-Q5_K_M.gguf | 1.17 GiB | Q5_K_M | high quality |
| Spark-X2.6-1.7B-Q4_K_M.gguf | 1.03 GiB | Q4_K_M | recommended default |
| Spark-X2.6-1.7B-Q3_K_M.gguf | 894 MiB | Q3_K_M | balanced |
| Spark-X2.6-1.7B-Q2_K.gguf | 754 MiB | Q2_K | smallest K-quant |
| Spark-X2.6-1.7B-IQ4_NL.gguf | 1011 MiB | IQ4_NL | i-quant |
| Spark-X2.6-1.7B-IQ4_XS.gguf | 971 MiB | IQ4_XS | i-quant |
| Spark-X2.6-1.7B-IQ3_M.gguf | 848 MiB | IQ3_M | i-quant, medium |
| Spark-X2.6-1.7B-IQ2_M.gguf | 678 MiB | IQ2_M | quality 2-bit |
| Spark-X2.6-1.7B-IQ2_XXS.gguf | 568 MiB | IQ2_XXS | smallest i-quant |
| Spark-imatrix.gguf | 3.7 MiB | imatrix | calibration data for requantization |
Sizes are file sizes on disk (GiB = 1024³ bytes, MiB = 1024² bytes).
Running it
Build llama.cpp
Any recent llama.cpp build with GGUF support. No custom code required — Spark2_5ForCausalLM is a standard architecture (hybrid sliding-attention / full-attention / short-conv layer types, Q/K-norm, gated FFN). The tokenizer and chat template are embedded in the GGUF.
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j --target llama-cli llama-server
Serve
./build/bin/llama-server \
-m Spark-X2.6-1.7B-Q4_K_M.gguf \
-ngl 999 -c 32768 --jinja \
--temp 1.0 --top-p 0.95 --top-k 0 --min-p 0
Web UI at http://127.0.0.1:8080, OpenAI-compatible API at /v1.
One-shot
./build/bin/llama-cli \
-m Spark-X2.6-1.7B-Q4_K_M.gguf \
-p "The capital of France is" -n 64 \
--temp 1.0 --top-p 0.95 --top-k 0 --min-p 0
About the imatrix
The shipped Spark-imatrix.gguf was computed from the model's own chat-template renders on a calibration corpus. To requantize using it:
llama-quantize --imatrix Spark-imatrix.gguf \
Spark-X2.6-1.7B-BF16.gguf \
Spark-X2.6-1.7B-Q4_K_M.gguf Q4_K_M
The imatrix is small (3.7 MiB) and architecture-agnostic; it can be reused for any quant of this checkpoint.
License
The model weights are licensed by XHToken under the Apache License 2.0. These quantizations are derivatives distributed under the same license. Base model: XHToken/Spark-X2.6-1.7B.