Ornith-1.5-9B DFlash GGUF
Official GGUF quantizations of ornith-ai/Ornith-1.5-9B-DFlash, the speculative decoding draft model designed to accelerate ornith-ai/Ornith-1.5-9B inference in llama.cpp.
Overview
This is a speculative decoding draft model (Block-Diffusion / DFlash architecture), not a standalone language model.
It runs alongside Ornith-1.5-9B GGUF to increase generation throughput by drafting up to 16 tokens per step (block_size = 16).
GGUF Files
| File | Quantization | Size |
|---|---|---|
| Ornith-1.5-9B-DFlash-Q4_K_M.gguf | Q4_K_M (Recommended) | 730.48 MiB |
| Ornith-1.5-9B-DFlash-Q6_K.gguf | Q6_K | 1021.29 MiB |
| Ornith-1.5-9B-DFlash-Q8_0.gguf | Q8_0 | 1319.66 MiB |
| Ornith-1.5-9B-DFlash-BF16.gguf | BF16 | 2474.66 MiB |
Usage
Pair this draft model with your target Ornith-1.5-9B model in llama.cpp using the -md (model draft) flag:
llama-cli \
-m Ornith-1.5-9B-Q4_K_M.gguf \
-md Ornith-1.5-9B-DFlash-Q4_K_M.gguf \
--spec-type draft-dflash \
-ngl 99 -c 4096 --temp 0.6 \
-p "<|im_start|>user\nWrite a quicksort in Python.<|im_end|>\n<|im_start|>assistant\n"
Attribution & License
- Distributed under the MIT License, matching upstream ornith-ai/Ornith-1.5-9B-DFlash.
- Target Model: ornith-ai/Ornith-1.5-9B.
- Reference GGUF release: ornith-ai/Ornith-1.5-9B-GGUF.