A llama.cpp-compatible GGUF of jzinno's DFlash2 speculative-decoding drafter for Ornith-1.5-35B-A3B — standalone conversion with the target's tokenizer cloned in and a reproducible conversion script.
Converted from the original safetensors checkpoint (jzinno/Ornith-1.5-35B-A3B-DFlash2, vLLM-only format) to the dflash GGUF architecture.
Priority note (corrected Sept 14): an earlier version of this card claimed to be the first llama.cpp GGUF of this drafter — wrong: gbuzhf's APEX repo has shipped Q8_0/BF16 conversions since Aug 26, 2026. This repo (published Sept 13) is an independent conversion; credit where due.
[PAD248077]Requires a llama.cpp build with DFlash2 support (PR #27816, b10896+ / v0.4.0):
llama-server -m Ornith-1.5-35B-A3B-IQ3_XXS-vcruz.gguf \
--spec-type draft-dflash \
--spec-draft-model Ornith-1.5-35B-A3B-DFlash2-draft-jzinno-F16.gguf \
--spec-draft-n-max 4 \
--cache-type-k q4_0 --cache-type-v q4_0 -ub 512
The cache/ubatch flags are the 16 GB recipe: they free ~1 GB so the target + drafter + KV fit in VRAM at large ctx. On 24 GB+ cards you can omit them. Pick one speculative path (DFlash2), never combine with draft-mtp.
A Q8_0 variant of the drafter (~0.5 GB smaller) is available in gbuzhf's APEX repo — we run it in production on v0.4.0 (loads and drafts normally in live traffic). On 16 GB cards the freed ~0.5 GB measurably delays the VRAM-overflow slowdown at large in-flight contexts.
Measured 2026-09-13, single-variable runs, every row confirmed by at least 2 independent runs:
| Allocated ctx | Default flags (KV q8_0, ub 2048) | Lean recipe (KV q4_0 + ub 512) |
|---|---|---|
| 2K | 318 tok/s ✓ fits | — |
| 16K | 316 tok/s ✓ fits | — |
| 32K | 50 tok/s ⚠️ VRAM overflow → WDDM paging | 328 tok/s ✓ fits |
| 50K | 53 tok/s ⚠️ VRAM overflow → WDDM paging | 326 tok/s ✓ fits |
The drafter wins (+70–75% over raw) at EVERY context size — the only failure mode is VRAM overflow. With default flags, target (~13.5 GB) + drafter (~1.1 GB) + KV + compute buffers cross the 16 GB line around ctx ≈ 20K, and Windows WDDM silently pages instead of crashing (draft rounds then cost ~61 ms instead of ~12.5 ms — the ⚠️ cells; the allocated size matters, not the token count in KV). Free ~1 GB with --cache-type-k q4_0 --cache-type-v q4_0 -ub 512 (already in the Usage command above) and the same 16 GB card runs 326 tok/s at 50K ctx. On 24 GB+ cards the default flags should sustain large ctx directly.
Accepted/step (llama-server mean len): 3.1–4.0 tokens per verification round; acceptance 53–74% (74% on short code-gen, ~53% on long-form), independent of the recipe.
Practical rule: watch your memory budget, not your context size. If target + drafter + KV + buffers fit, use the drafter; if they overflow, slim the recipe (KV q4_0, smaller -ub) or stay raw.
Corrections (2026-09-13, same day). v1 of this card reported 160/135/57 tok/s from ad-hoc runs with uncontrolled flags — superseded. v2 reported the speeds above but attributed the large-ctx drop to a "kernel regime cliff" and advised keeping ctx ≤ 16384 — wrong: it was plain VRAM overflow (silent WDDM paging), fully removed by the lean recipe. This version also restructures the table by recipe so it can't be skimmed as "useless above 16K" — the drafter is useful at every ctx it fits in VRAM.
-ub) or stay raw.Reports welcome — open a discussion with your GPU, --ctx-size, KV recipe and numbers.
For reference, the source checkpoint reports accepted lengths up to 4.87 (n=7) on H200/vLLM against both BF16 and NVFP4 targets.
ne from the reversed numpy shape — do not transpose)tokenizer.ggml.mask_token_id = 248077 (from dflash_config.mask_token_id in the source config.json)*_conv_base tensors stored in F32 (binary-op operands in the dflash graph require matching F32 types; F16 triggers a binbcast assert on CUDA / "unsupported types f32/f16" on CPU)convert_from_safetensors.py in this repo