Soulfate24/Spark-X2.6-1.7B-DSpark-Paretrix-GGUF

🧮 Paretrix GGUF quantizations of XHToken/Spark-X2.6-1.7B with DSpark draft

⚡ DSpark drafts

Three speculative drafts ship with the rungs. The official draft is the reference; the two others are rebuilt by 16_dspark-generator.py, each for one purpose. They differ by their body and by their readout: the official draft scores the whole head of the target, the two others score only the 32,768 tokens of their own d2t table, the ones Spark-X2.6-1.7B writes most.

Every draft quantized Paretrix-Balanced, against Precision-42pc ⭐; a 16,000-token prefix (-c 16384), the chat template on every probe, draft KV q8_0, --spec-draft-n-max 5, llama.cpp build 12283, RTX 4060 8 GB; each speedup against the target alone at the same KV type (about 104 t/s at q8_0, 88 t/s at q5_1). Output share: the generated tokens that the readout covers, the ceiling of its acceptance.

• draft-32K — the speed pick. The same layers as the official draft, hence the same proposals: at KV q8_0 they accept 243 and 245 drafted tokens. What changes is the readout: the official draft scores all 131,072 tokens of the target's head on every draft pass; draft-32K scores only the 32,768 tokens that make up 99.45% of what Spark-X2.6-1.7B writes, through a d2t table of its own. On those tokens its logits are the official draft's, and the speedup goes from ×1.42 to ×1.54 at KV q8_0 (×1.62 to ×1.85 at q5_1): on a small target, the vocab head is the largest share of every draft pass. dspark-32K-BF16.gguf is the source for another quantization.
• draft-lite — the speed of draft-32K, the lightest. The official draft pruned to the 4 layers that score best (FFN 1536), with the same d2t readout as draft-32K, then trained for a night on non-causal blocks of 3 to 5 tokens: 161.3 t/s at KV q8_0, one token per second behind draft-32K for 116 MiB less, and faster than the official draft at both KV types (×1.56 and ×1.77) for 89 MiB less. In the Paretrix sub-agent bench (windows of 131K to 524K tokens, KV q5_1), it doubles the decoding speed of the whole battery: 47.6 → 98.9 t/s. dspark.safetensors and dspark_config.json hold its trained fp32 weights, the starting point to continue training or to export again; dspark-lite-BF16.gguf is the source for another quantization.
• The official draft stays the reference, quantized like every Paretrix module.

The drafts run on a llama.cpp build with Spark2.5 and DSpark support (XHToken's fork), next to any rung:

llama-server -m model-Paretrix-Precision-42pc.gguf -md draft-32K/dspark-32K-Paretrix-Balanced.gguf \
  --spec-type draft-dspark --spec-draft-n-max 5 --spec-draft-p-min 0 -fa on -ctkd f16 -ctvd f16

Keep the target KV cache in f16 while it fits: on a quantized cache, every verified block re-converts the whole cache, a cost that grows with the context. Beyond, a quantized cache still pays: ×1.54 at q8_0 and ×1.85 at q5_1 with a 16K prefix (table). q5_1 slows the target alone more than the pair, and both reach the same decode speed with draft-32K (162 t/s). The draft KV stays small (a 512-token window, a few MiB).

ℹ️ About Paretrix Quantization Suite

Paretrix is an empirical, activation-aware quantization suite for GGUF language models and speculative decoding modules.

Pareto + Matrix → Paretrix. Each rung is planned so that no better trade exists at its weight among the options measured. Around three stock bases (Q8_0, Q6_K-imx, IQ4_XS-imx), every tensor class is moved alone to every type and measured against the BF16 logits; an exact knapsack then plans each rung from those cells, and a duel at the verdict regime decides whether the plan replaces the champion. What one model measures, the next model of its family inherits.

Standard uniform quantization applies one bitwidth family across the whole network. Paretrix reads the model's own response to each move (which classes are expensive to cut, which are nearly free, which pay back a buy), then spends the budget where the measurements show the greatest return.