Ornith-1.5-122B-DFlash2-GGUF

This model was converted to GGUF format from incoai/Ornith-1.5-122B-DFlash2 using upstream llama.cpp (ead9f32).

This is a DFlash 2 draft model (0.9B, block-diffusion drafter) for ornith-ai/Ornith-1.5-122B. It is not a standalone model. It reuses the target's token embeddings and LM head at runtime, so load it with -md next to an Ornith-1.5-122B GGUF.

Quants

The following quants are available: BF16, F16, F32, IQ3_M, IQ3_S, IQ4_NL, IQ4_XS, MXFP4_MOE, Q1_0, Q2_0, Q2_K, Q3_K, Q3_K_L, Q3_K_M, Q3_K_S, Q4_0, Q4_1, Q4_K, Q4_K_M, Q4_K_S, Q5_0, Q5_1, Q5_K, Q5_K_M, Q5_K_S, Q6_K, Q8_0, TQ1_0, TQ2_0

IQ1/IQ2/IQ3_XXS/IQ3_XS/Q2_K_S are not provided (they need an importance matrix). Q4_K = Q4_K_M and Q3_K = Q3_K_M; MXFP4_MOE equals Q8_0 here (no experts). Drafter quant barely matters (Q8_0, Q5_K_M and Q4_K_M benchmark within 1%), so Q4_K_M is a good default. Draft length matters much more; see Benchmarks.

Usage

Requires a llama.cpp build with DFlash 2 support (current master).

llama-server -m Ornith-1.5-122B-UD-IQ3_XXS.gguf \
  -md Ornith-1.5-122B-DFlash2-Q4_K_M.gguf \
  --spec-type draft-dflash --spec-draft-n-max 4

Benchmarks

Target unsloth/Ornith-1.5-122B-GGUF UD-IQ3_XXS on an AMD Strix Halo (Radeon 8060S, 96 GB VRAM, llama.cpp Vulkan ead9f32), 16k context. Greedy decoding, thinking off, 300 max tokens, mean of 3 prompts (code, short story, math). Each n-max is set at server start; the per-request speculative.n_max override was ignored in this build.

Baseline without a drafter: 27.0 tok/s.

`--spec-draft-n-max`	Q8_0	Q5_K_M	Q4_K_M	Acceptance
4	33.7	33.7	33.4	~30%
5	29.3	30.7	29.8	~23%
6	27.4	29.3	27.1	~19%
7	24.7	25.0	24.7	~16%
8	12.2	12.8	12.8	~13%
10	10.9	11.4	10.8	~11%
15	8.8	9.6	8.8	~7%
17	8.8	9.7	8.8	~6%

Use n-max 4. It's the fastest setting tested (about 1.25× the baseline on average); 5 and 6 are only marginally faster than no drafter. Long drafts are slower than no drafter on this hardware.
Speed depends on content. At n-max 4, code runs at about 39 tok/s and math at about 42 (1.45–1.6×). Free-form prose drops to about 19–20 tok/s, slower than the baseline.
Speed halves going from n-max 7 to 8. This looks like a backend limit rather than worse drafts; it may differ on CUDA.
These numbers are from one machine and a low-bit target; tune n-max on your own hardware.

Reproduce

python convert_hf_to_gguf.py Ornith-1.5-122B-DFlash2 --target-model-dir Ornith-1.5-122B --outtype bf16 --outfile Ornith-1.5-122B-DFlash2-BF16.gguf
llama-quantize Ornith-1.5-122B-DFlash2-BF16.gguf Ornith-1.5-122B-DFlash2-Q8_0.gguf Q8_0

(--target-model-dir only needs the target's config and tokenizer files.)