Soulfate24/Spark-X2.6-4B-DSpark-Paretrix-GGUF

🧮 Paretrix GGUF quantizations of XHToken/Spark-X2.6-4B with DSpark draft

⚡ DSpark drafts

Three speculative drafts ship with the rungs. The official draft is the reference; the two others are rebuilt by 16_dspark-generator.py, each for one purpose. They differ by their body and by their readout: the official draft scores the whole head of the target, the two others score only the 32,768 tokens of their own d2t table, the ones Spark-X2.6-4B writes most.

Every draft quantized Paretrix-Balanced, against Quality-36pc ⭐; a 16,000-token prefix (-c 16384), the chat template on every probe, draft KV q8_0, --spec-draft-n-max 5, llama.cpp build 12283, RTX 4060 8 GB; each speedup against the target alone at the same KV type (about 55.6 t/s at q8_0, 42.9 t/s at q5_1). Output share: the generated tokens of 60 held-out trajectories that the readout covers, the ceiling of its acceptance.

• draft-32K — the speed pick. The same layers as the official draft, hence the same proposals: both accept the same 207 of 359 drafted tokens at KV q8_0. What changes is the readout: the official draft scores all 131,072 tokens of the target's head on every draft pass; draft-32K scores only the 32,768 tokens that make up 99.45% of what Spark-X2.6-4B writes, through a d2t table of its own. On those tokens its logits are the official draft's, every draft pass is cheaper, and the speedup goes from ×1.92 to ×2.02 at KV q8_0 (×2.30 to ×2.46 at q5_1). On an agent's work (below) the lead widens: ×3.1 to ×3.3 over the target alone at every tier, against ×2.4 to ×2.6 for draft-lite. dspark-32K-BF16.gguf is the source for another quantization.
• draft-lite — the VRAM pick. The official draft pruned to the 4 layers that score best (FFN 10240 → 1536), with the same d2t readout as draft-32K, then trained for a night on non-causal blocks of 3 to 5 tokens, at every length it serves: ×1.78 at KV q8_0 and ×2.30 at q5_1, the official draft's speedup at q5_1 for half its size (209 MiB against 415). It leaves 244 MiB more VRAM than draft-32K: room for a longer context, the pick for a GPU under 8 GB. It replaces the earlier Lites: one trained from the target's own layers (mean accepted run τ 2.73 in training), then one with a 16K readout (τ 3.03), which this one (τ 3.18) outruns by 16% on an agent's work, where identifiers and line numbers fall outside 16,384 tokens. dspark.safetensors and dspark_config.json hold its trained fp32 weights, the starting point to continue training or to export again; dspark-lite-BF16.gguf is the source for another quantization.
• The official draft stays the reference, quantized like every Paretrix module. Its acceptance follows the rung it drafts for: 57.7% of its drafted tokens on Quality-36pc, 42.9% on Compact-33pc (KV q8_0).

The drafts run on a llama.cpp build with Spark2.5 and DSpark support (XHToken's fork), next to any rung:

llama-server -m model-Paretrix-Quality-36pc.gguf -md draft-32K/dspark-32K-Paretrix-Balanced.gguf \
  --spec-type draft-dspark --spec-draft-n-max 5 --spec-draft-p-min 0 -fa on -ctkd f16 -ctvd f16

Keep the target KV cache in f16 while it fits (36 KiB per token: about 64–70K tokens beside a rung on 8 GB): on a quantized cache, every verified block re-converts the whole cache, a cost that grows with the context. Beyond, a quantized cache still pays: ×2.02 at q8_0 and ×2.46 at q5_1 with a 16K prefix (table), ×1.51 at 64K with q8_0. q5_1 slows the target alone more than the pair, so its speedups read higher while the fastest decode stays at q8_0. The draft KV stays small (a 512-token window, a few MiB).

ℹ️ About Paretrix Quantization Suite

Paretrix is an empirical, activation-aware quantization suite for GGUF language models and speculative decoding modules.

Pareto + Matrix → Paretrix. Each rung is planned so that no better trade exists at its weight among the options measured. Around three stock bases (Q8_0, Q6_K-imx, IQ4_XS-imx), every tensor class is moved alone to every type and measured against the BF16 logits; an exact knapsack then plans each rung from those cells, and a duel at the verdict regime decides whether the plan replaces the champion. What one model measures, the next model of its family inherits.

Standard uniform quantization applies one bitwidth family across the whole network. Paretrix reads the model's own response to each move (which classes are expensive to cut, which are nearly free, which pay back a buy), then spends the budget where the measurements show the greatest return.