This repository provides GGUF files of mradermacher/OxCoder-9B-i1-GGUF
with an embedded MTP (Multi-Token Prediction) block, ready for speculative
decoding in llama.cpp with --spec-type draft-mtp.
OxCoder-9B (OrionLLM/OxCoder-9B) is a Qwen3.5-9B-based coding model that ships without MTP tensors. The MTP block was transplanted (copied tensor by tensor, with no training) from a file of a4lg/Qwen3.5-9B-MTP-ONLY-GGUF, itself extracted from the original Qwen3.5-9B.
OxCoder-9B.i1-*.gguf by
mradermacher,
copied tensor by tensor, unchanged.blk.32.*): taken from a file of
a4lg/Qwen3.5-9B-MTP-ONLY-GGUF,
an MTP-only subset of Qwen/Qwen3.5-9B.
The quant of the donor file used for each file is listed in the table under "Files".token_embd and output
with OxCoder, so each file is only slightly larger than the original
quant and uses less VRAM than loading a separate draft model.| File | Quant main | Quant MTP | Size |
|---|---|---|---|
| OxCoder-9B.i1-MTP-IQ1_S.gguf | i1-IQ1_S | Q4_K_M | 2.7 GiB (~2.9 GB) |
| OxCoder-9B.i1-MTP-IQ1_M.gguf | i1-IQ1_M | Q4_K_M | 2.8 GiB (~3.0 GB) |
| OxCoder-9B.i1-MTP-IQ2_XXS.gguf | i1-IQ2_XXS | Q4_K_M | 3.0 GiB (~3.3 GB) |
| OxCoder-9B.i1-MTP-IQ2_XS.gguf | i1-IQ2_XS | Q4_K_M | 3.2 GiB (~3.4 GB) |
| OxCoder-9B.i1-MTP-IQ2_S.gguf | i1-IQ2_S | Q4_K_M | 3.3 GiB (~3.6 GB) |
| OxCoder-9B.i1-MTP-IQ2_M.gguf | i1-IQ2_M | Q4_K_M | 3.5 GiB (~3.8 GB) |
| OxCoder-9B.i1-MTP-Q2_K_S.gguf | i1-Q2_K_S | Q4_K_M | 3.6 GiB (~3.9 GB) |
| OxCoder-9B.i1-MTP-Q2_K.gguf | i1-Q2_K | Q4_K_M | 3.7 GiB (~4.0 GB) |
| OxCoder-9B.i1-MTP-IQ3_XXS.gguf | i1-IQ3_XXS | Q4_K_M | 3.8 GiB (~4.1 GB) |
| OxCoder-9B.i1-MTP-IQ3_XS.gguf | i1-IQ3_XS | Q4_K_M | 4.1 GiB (~4.4 GB) |
| OxCoder-9B.i1-MTP-Q3_K_S.gguf | i1-Q3_K_S | Q4_K_M | 4.1 GiB (~4.4 GB) |
| OxCoder-9B.i1-MTP-IQ3_S.gguf | i1-IQ3_S | Q4_K_M | 4.2 GiB (~4.5 GB) |
| OxCoder-9B.i1-MTP-IQ3_M.gguf | i1-IQ3_M | Q4_K_M | 4.3 GiB (~4.6 GB) |
| OxCoder-9B.i1-MTP-Q3_K_M.gguf | i1-Q3_K_M | Q4_K_M | 4.5 GiB (~4.8 GB) |
| OxCoder-9B.i1-MTP-Q3_K_L.gguf | i1-Q3_K_L | Q4_K_M | 4.7 GiB (~5.1 GB) |
| OxCoder-9B.i1-MTP-IQ4_XS.gguf | i1-IQ4_XS | Q4_K_M | 5.0 GiB (~5.4 GB) |
| OxCoder-9B.i1-MTP-Q4_0.gguf | i1-Q4_0 | Q4_K_M | 5.1 GiB (~5.5 GB) |
| OxCoder-9B.i1-MTP-Q4_K_S.gguf | i1-Q4_K_S | Q4_K_M | 5.1 GiB (~5.5 GB) |
| OxCoder-9B.i1-MTP-IQ4_NL.gguf | i1-IQ4_NL | Q4_K_M | 5.2 GiB (~5.6 GB) |
| OxCoder-9B.i1-MTP-Q4_K_M.gguf | i1-Q4_K_M | Q4_K_M | 5.4 GiB (~5.8 GB) |
| OxCoder-9B.i1-MTP-Q4_1.gguf | i1-Q4_1 | Q4_K_M | 5.6 GiB (~6.0 GB) |
| OxCoder-9B.i1-MTP-Q5_K_S.gguf | i1-Q5_K_S | Q5_K_M | 6.0 GiB (~6.5 GB) |
| OxCoder-9B.i1-MTP-Q5_K_M.gguf | i1-Q5_K_M | Q5_K_M | 6.2 GiB (~6.7 GB) |
| OxCoder-9B.i1-MTP-Q6_K.gguf | i1-Q6_K | Q6_K | 7.1 GiB (~7.6 GB) |
The MTP block adds about 0.2 GiB.
[!note] Vision projectors (mmproj) are included. They were taken from mradermacher/OxCoder-9B-GGUF and were trained for this model.
Requires a llama.cpp build with MTP draft support (--spec-type draft-mtp).
# Replace <QUANT> with the quant of the file you downloaded
llama-server \
-m OxCoder-9B.i1-MTP-<QUANT>.gguf \
-ngl auto -c 0 -fit on -fitc 32768 -fa on -np 1 \
-ctk q8_0 -ctv q8_0 \
--temp 0.6 --top-k 20 --min-p 0.0 \
--spec-type draft-mtp --spec-draft-n-max 2
Use the grafted file as the main model. Do not pass it again with
--model-draft.
Notes:
--spec-draft-n-max defaults to 3. In my tests 2 was
consistently a little better than 3; values of 1 or 4 and above were slower
(see below).-ub 512). With
-ub 1024 or -ub 2048, generation with MTP dropped to ~22 t/s,
probably because the larger compute buffer left less VRAM for the
model layers under -fit on.The MTP block was transplanted with
convert.py
by @buzz, following "Method 2" of the
a4lg README:
# ./convert.py TARGET DONOR OUTPUT
python3 convert.py \
OxCoder-9B.i1-<QUANT>.gguf \
Qwen3.5-9B-MTP-ONLY-<DONOR_QUANT>.gguf \
OxCoder-9B.i1-MTP-<QUANT>.gguf
The script as published did not work for the qwen35 architecture.
It raises block_count from 32 to 33 but copies per-layer array keys
unchanged, so llama.cpp refused the output with:
key qwen35.attention.recurrent_layers has wrong array length; expected 33, got 32
The donor file does not carry that key, so I extended per-layer arrays
in the loop that writes the target's KVs, appending False for the MTP
block (it is a full-attention layer: attn_q/k/v/output, no ssm_*):
contents = field.contents()
# Per-layer arrays have one entry per block: add an entry for the MTP block
if len(contents) == target_block_count:
src_field = source_reader.get_field(key)
if src_field is not None and len(src_field.contents()) == source_block_count:
extra = list(src_field.contents())[target_block_count:]
elif key == f"{arch}.attention.recurrent_layers":
# Donor lacks the key; the MTP block uses full attention
extra = [False]
else:
extra = []
contents = list(contents) + extra
if sub_type == GGUFValueType.BOOL:
contents = [int(x) for x in contents]
write_array_value(fout, sub_type, contents)
Resulting metadata:
qwen35.block_count = 33
qwen35.nextn_predict_layers = 1
qwen35.attention.recurrent_layers = [... 33 entries, last one False]
Updated script convert.py.
All GGUF files in this repository are distributed under the Apache License, Version 2.0, the license of both Qwen/Qwen3.5-9B and OrionLLM/OxCoder-9B. A copy of the license is included in the LICENSE file.
Copyright 2026 Alibaba Cloud (Qwen3.5-9B, including the MTP head)
Modification notice: the files were produced by mechanically combining tensors from the two GGUF files listed above and adjusting the metadata described in "How it was made". No weights were changed or trained.
This README file is licensed under the terms of CC-BY-4.0. It is partly based on the README of a4lg/Qwen3.5-9B-MTP-ONLY-GGUF (CC-BY-4.0, Copyright 2026 a4lg).
Copyright 2026 Daniel Dias Rodrigues (Nerun).
convert.py MTP transplant scriptmradermacher/OxCoder-9B-i1-GGUF