Spark-X2.6-4B — GGUF

Full-precision GGUF conversion of XHToken/Spark-X2.6-4B, plus Q8_0 / Q6_K / Q5_K_M / Q4_K_M built with the same recipe as Myric/Spark-X2.5-4B-GGUF.

X2.6 keeps the exact Spark-X2.5 architecture — same 36 layers, tensor names and shapes — and gains capability through continued training (the model card's own description). The weights are not a light adapter: per-tensor relative L2 against X2.5 is ~0.22–0.35 on the linear layers, so treat this as a new checkpoint, not a patch.

⚠️ Requires a patched llama.cpp — upstream will NOT load this

spark2_5 is not in upstream llama.cpp. Loading this file with a stock build fails at the architecture check. Build from the branch that adds it:

git clone -b model/Spark2_5 https://github.com/brywil/llama.cpp
cd llama.cpp && cmake -B build -DGGML_CUDA=ON && cmake --build build -j
./build/bin/llama-cli -m Spark-X2.6-4B-BF16.gguf --ctx-size 32768 -ngl 999 -p "..."

Based on upstream 6d0549831. The branch adds the arch, a converter, and a new BPE pre-tokenizer type; nothing else is changed. X2.6 converts and loads with that branch unmodified — its config.json, modeling_spark.py, configuration_spark.py, tensor names and shapes are identical to X2.5.

Always pass --ctx-size. The model declares max_position_embeddings: 1048576, and llama.cpp defaults to the trained context — that will try to allocate the full 1 M window.

What's here

All 362 tensors present in every file, attn_gate F32 in all of them, and zero block-32 substitutions — verified by reading the type histogram out of the shipped bytes rather than trusting the filename.

The bf16 is published deliberately: the arch needs a fork to run at all, so it is the reference artifact to check a quant against.

Which one

Q5_K_M is the default recommendation — ~2.75 GiB fits an 8 GB card with room for real context, and 128k of KV costs only another 4.5 GiB. Q4_K_M if you are tight on VRAM. Q8_0 if you want a near-reference artifact without carrying 7.67 GiB.

The imatrix was built from the same code-inclusive mixed corpus as X2.5 (inkling_calib_mixed.txt), 1,000 chunks × 512 tokens = 512k tokens.

The model

4.11 B dense, 36 layers, apache-2.0. Interleaved sliding-window attention — 27 sliding (window 512) and 9 full, in a strict sssF pattern. 16 query heads, 4 KV heads, head_dim 256, hidden 2560, GELU FFN, 131,072 vocab, tied embeddings. It is a reasoning model and emits a thinking block.

X2.6's chat template adds reasoning_effort (high default, low) to the system prompt and supports three thinking levels — none / low / high. The default system prompt therefore carries extra instruction text that X2.5's did not; output-token counts are not directly comparable across the two model cards even at an equal score. The template also emits compact JSON for tool definitions (separators=(',', ':')).

Long context is cheap here, by design

Only the 9 full-attention layers scale with context; the 27 sliding layers are bounded by their 512 window.

9 layers x 4 KV heads x 256 head_dim x 2 (K+V) x 2 bytes = 36 KiB/token

So 128k of context costs less KV than the weights do. Note this is a memory calculation, not a retrieval claim — long-context quality is unmeasured here.

Notes for anyone quantizing this

• Every dimension is divisible by 256 (hidden 2560, intermediate 10240, vocab 131072, KV dim 1024), so all QK_K types are legal. A file labelled Q4_K_M really will be Q4_K_M — unlike archs with an awkward dimension, where llama-quantize silently substitutes block-32 types.

• Pin attn_gate to F32. g_proj is Linear(2560, 16) — 40,960 params per layer, 1.47 M total, about 5.9 MB. Its sigmoid output multiplies the attention result, so error there compounds along the sequence instead of adding to one layer's output, which is exactly the case where per-tensor weight error under-reports the true cost. At this size the bits are free. The converter emits it as F32 — but llama-quantize will still quantize it unasked, because ne0 is 2560 and therefore 256-divisible, so Q5_K is perfectly legal on [2560,16]. Legal is not correct here. Pass it explicitly:

  ./build/bin/llama-quantize --imatrix spark-x26-4b.imatrix --tensor-type attn_gate=f32 \
      Spark-X2.6-4B-BF16.gguf out.gguf Q5_K_M
  

  Cost: 5.9 MB on a 2.7 GiB file. Every quant here was built that way.

Verification

Converted with the model/Spark2_5 branch at 498cbc7e0 (upstream base 6d0549831), then checked by reading the shipped bytes:

• 362 tensors in BF16 and in every quant, ne0 % 256 == 0 for every tensor — zero block-32 substitutions;
• attn_gate F32 in all 36 layers of every tier;
• GGUF metadata carries both rope configs (sliding: 256/256 dims at theta 10,000; full: 64/256 at theta 5,000,000), 36 layers, 16/4 heads, head_dim 256, context 1,048,576, sliding window 512.

The forward itself is the X2.5 port's, whose greedy output was verified byte-identical to the HF reference (transformers 4.57.1) over 24 tokens; X2.6 uses the same modeling_spark.py and the same tensor layout, so no new port work was needed. X2.6 has not been independently checked against transformers here — that is stated rather than implied.

Measured

Agentic coding, 42 tasks (Q6_K)

Same harness, box and config as the X2.5 comparison (single slot, --timeout 900, gated backoff 900,1800,3600, 32,768-token output cap, temperature 1.0). Driven through a real opencode agentic loop with tool calling, graded by a fresh pytest per task.

Mean output tokens over valid tasks:

Every void in both arms is the same harness limit: one step truncated at the 32,768-token cap with no degeneration detected. X2.5 hit it 12 times; X2.6 three times.

On the 30 tasks valid and passed in both arms, X2.6 produced 77,917 output tokens against X2.5's 362,109 — 0.22×. One X2.5 task (serialize_binary_tree) accounts for 131,200 of those; excluding it the ratio is 0.31×. Treat the token figure as directional, not a measurement: this is one rep per arm, and this repo has measured a 6.8× run-to-run spread on token counts. The score and void counts are the solid comparison; a token ratio needs reps to quote.

Token counts are also not perfectly like-for-like across the two cards because X2.6's template injects reasoning_effort text into the system prompt. The X2.5 repo's published agentic table remains the separate bf16 run from 2026-09-08 (32/32 gradeable); this table is the same-box, same-tier comparison that did not exist before.

KL divergence vs this repo's BF16

30 chunks, one binary, wiki.test.raw (sha256 173c87a5…dd08), -c 2048 -ngl 99 -fa on -t 20. BF16 PPL over the same 30 chunks: 17.2396.

same top-1 % is the figure worth quoting. The ladder lands within half a point of the X2.5 quants (Q8_0 97.18, Q6_K 93.64), as it should — same architecture, same recipe, same corpus. Perplexity is deliberately not used to pick a tier on this arch: 27 of 36 layers see only a 512-token window, and the X2.5 PPL ladder came out non-monotonic with three quants "beating" BF16 at a zero noise floor.

Throughput (thor: Xeon E5-2670 v2, RTX 4060 Ti 16 GB)

llama-bench, Q6_K, -ngl 99 -fa on, 3 reps, 512-token prompt / 128-token generation:

Single stream, discrete card. Do not compare this row with the X2.5 repo's GB10 figures — that is unified memory and a different box.

Provenance

Quantized with llama-quantize from the model/Spark2_5 branch at 498cbc7e0 (upstream base 6d0549831), x86_64 CUDA build verified to contain the arch before use. Every file was loaded and generated from before upload; the verification section is that evidence.

Recipe and provenance notes: brywil/gguf-quant-lab → benchmark_results/spark-x26-4b_QUANTS.md (commit d4d757b).

Credit

Model by XHToken. This repo is a format conversion only — no weights were retrained or altered.