Ornith-1.5-35B-A3B-BigBang-MTP — GGUF

GGUF quants of EryriLabs/Ornith-1.5-35B-A3B-BigBang-MTP: a TIES merge of Ornith-1.5-35B-A3B + BigBang-v1 over Qwen3.6-35B-A3B, with Ornith's untrained placeholder MTP head replaced by Qwen3.6's trained one — so speculative decoding actually works (~75% draft acceptance vs ~13% for the stock random head; see the main repo card for the measurements).

Requires a recent llama.cpp (b10507+ tested; the Qwen3.5-MoE hybrid arch is new).

Where is the MTP?

Two options, pick one:

1. Separate draft (recommended): the main GGUFs deliberately contain no MTP tensors. Download mtpdraft-Q8_0.gguf (2 GB) once and pass it with -md — it works with every quant here, and with stock Ornith-1.5 GGUFs too.
2. Embedded: Ornith-1.5-35B-A3B-BigBang-MTP-Q4_K_M-embedded.gguf carries the trained head inside the main file (stock-style blk.40 layout), for tools that expect it embedded.

If you inspect a non-embedded main GGUF and see no MTP tensors, that is by design, not a missing feature.

Files

(BF16 reference weights live in the main safetensors repo.)

Usage

The MTP draft is optional. The main GGUFs contain no MTP tensors and work as normal models on their own (LM Studio, Ollama, plain llama.cpp):

llama-server -m Ornith-1.5-35B-A3B-BigBang-MTP-Q4_K_M.gguf -ngl 99 -c 4096 --jinja -fa on

Add the draft for ~+29% generation speed (outputs are unchanged; speculative decoding only affects speed):

llama-server -m Ornith-1.5-35B-A3B-BigBang-MTP-Q4_K_M.gguf \
  -md mtpdraft-Q8_0.gguf --spec-draft-n-max 4 --spec-draft-n-min 1 \
  -ngl 99 -c 4096 --jinja -fa on

mtpdraft-Q8_0.gguf also works as a -md draft for stock Ornith-1.5 GGUFs (same tower).

Reasoning is always on (<think>), so allow generous max_tokens. Suggested sampling: temp 0.1, top_k 50.

On a single RTX 3090 (Q4_K_M + Q8_0 draft) the draft lifts generation from 131 to 169 tok/s (+29%), with 55–75% acceptance (highest on code).

Merged and measured by EryriLabs (Dwain Barnes), 2026-08-20.