gemma4-26b-a4b-it-dspark-multidomain (GGUF)

GGUF export of systalyze/gemma4-26b-a4b-it-dspark-multidomain, a DSpark speculative draft for google/gemma-4-26B-A4B-it, for use with llama.cpp's draft-dspark speculative decoding.

This is a draft model. It cannot generate text on its own: it reads hidden states from target layers [3, 10, 18, 25, 28] and proposes 7 tokens per step for the target to verify.

Files

You also need a GGUF of the target, google/gemma-4-26B-A4B-it (we tested with Q8_0).

Usage

llama-server \
  -m gemma-4-26B-A4B-it-Q8_0.gguf \
  -md gemma4-26b-a4b-it-dspark-multidomain-bf16.gguf \
  --spec-type draft-dspark --spec-draft-n-max 7 \
  -ngl 999 -ngld 999 -fa on --jinja

--spec-draft-n-max is clamped to the trained block size (7).

llama.cpp version. This GGUF loads on current upstream llama.cpp. The draft was trained with final logit softcapping (30.0). Upstream llama.cpp does not yet apply it for DSpark drafts, which lowers acceptance length by about 0.07 (about 2%) in our tests. It still works correctly: the target verifies every token, so output quality is unchanged.

Acceptance length

Acceptance length is the number of tokens produced per draft step: the accepted draft tokens plus one token from the target.

Test setup. We measured on a single H200 with llama.cpp (CUDA build, with logit softcapping applied), using the Q8_0 target and the bf16 draft. Each request generated 384 tokens, with 8 prompts per domain. The temperature 1.0 runs use 3 seeds and no top-k, top-p or min-p truncation.

License

Apache-2.0, same as the base draft and google/gemma-4-26B-A4B-it.