Spark-X2.6-4B-2GB-GGUF (imatrix-calibrated IQ3_M + IQ4_XS)

Two imatrix-calibrated quantizations of XHToken/Spark-X2.6-4B, selected from a full quantization ladder evaluated with complete perplexity runs and real-chat capability checks on llama.cpp:

• IQ3_M (3.66 BPW, 2.04 GB) — the balanced pick, and the one to choose for Chinese-heavy use: it produced a fully correct quicksort in the zh code check, including equal-element handling.
• IQ4_XS (4.69 BPW, 2.38 GB) — the quality-first pick: perplexity statistically identical to the upstream Q4_K_M at two-thirds the size. Caveat: in the zh code check its quicksort had subtle ordering bugs in both thinking modes (pivot misplacement / dropped duplicates — see below), so prefer IQ3_M when Chinese code generation matters.

Files

That is the complete list: GGUF weights only — no safetensors, no calibration dataset text, no eval harness.

SHA256: fa91f118b3228766ac518dfed320094c9e7439775bf3d1168d93a7049ec72fbb (IQ3_M), f309c71ce5d0a22860628cae512115a9b0138e2bff0b12c5b34753223c0d684a (IQ4_XS)

Why these two? (the ladder, measured)

All perplexities below are complete wikitext-2-raw-test runs (512-token chunks, llama.cpp b393, 24 CPU threads), not sampled estimates:

IQ3_XXS is statistically tied with IQ3_M on PPL but failed the real-chat Chinese check (ran out of its thinking budget mid-answer and produced a duplicate-dropping quicksort), so IQ3_M is the lowest bit-width that actually preserves this model's capability.

Imatrix provenance

Built with llama-imatrix (512-token context) on a ~113K-token slice, round-robin balanced from: wikitext-2-raw validation, bartowski's calibration_datav5 (Dampf enhancedV3 + Kalomaze + edaddario), 13 chapters of the d2l-zh Chinese deep-learning textbook (25% zh density), and C++/Python sources from llama.cpp. The corpus was paragraph-filtered against a safety blocklist (adult, gambling, SEO/pharma spam, weapons/drugs, malware, slurs) — 17 paragraphs dropped, counts only. Final slice: 12.2% Chinese characters (Spark-X2.6 is an en/zh model; a first imatrix built without zh measured worse on every rung — e.g. IQ4_XS 29.55 vs 27.97 here).

Real-chat verification (llama.cpp b393, greedy, both quants)

Both quants share the upstream thinking-verbosity trait (identical on F16): a 500–900-token budget can be consumed by the [Start thinking] block. Disable thinking for short direct answers (--chat-template-kwargs '{"enable_thinking": false}' in llama.cpp, or "chat_template_kwargs": {"enable_thinking": false} on OpenAI-compatible servers) and give thinking-mode generations a generous max_tokens.

Bottom line: IQ4_XS is the pick for maximum quality (English-heavy workloads, best PPL below 3.5 GB); IQ3_M is the pick when size matters most or when Chinese code generation is part of the job — it was the only low-bit rung whose zh code survived simulation testing.

Known upstream trait (not quantization damage)

Spark-X2.6's chat template defaults to high thinking-effort, which is verbose: with thinking enabled, a 500–900-token budget can be consumed entirely by the [Start thinking] block before the final answer — this is identical on the unquantized F16. For short direct answers, disable thinking per request: --chat-template-kwargs '{"enable_thinking": false}' in llama.cpp, or "chat_template_kwargs": {"enable_thinking": false} on OpenAI-compatible servers; and give thinking-mode generations a generous max_tokens.

Usage

llama.cpp b393+ (spark2_5 arch):

# quality-first
llama-cli -m Spark-X2.6-4B-IQ4_XS.gguf -c 32768 -t 16 -cnv

# size-first
llama-cli -m Spark-X2.6-4B-2GB-IQ3_M.gguf -c 32768 -t 16 -cnv

# short answers without the thinking phase:
llama-cli -m Spark-X2.6-4B-2GB-IQ3_M.gguf -c 32768 -t 16 \
  --chat-template-kwargs '{"enable_thinking": false}'

Re-quantize to any other width using the shipped imatrix:

llama-quantize --imatrix spark-x2.6-2gb.imatrix \
  Spark-X2.6-4B-F16.gguf your-choice.gguf IQ4_XS

Provenance

Originally published on ModelScope as Dalvlad/Spark-X2.6-4B-LowBit-GGUF. This Hugging Face copy is renamed to Spark-X2.6-4B-2GB-GGUF; the weights and imatrix are byte-identical to that release (same SHA256), only the filenames and links differ.

Credits

• XHToken/Spark-X2.6-4B — the upstream model (Apache-2.0). This repo re-quantizes the upstream F16 GGUF; all capability is theirs.
• Calibration corpus: bartowski's calibration_datav5, wikitext-2, d2l-zh, llama.cpp sources.
• Methodology: llama.cpp imatrix discussion #5263.