OxCoder-9B — ROCmFP4 for AMD Strix Halo (gfx1151)

ROCmFP4 / ROCmFPX quantizations of OrionLLM/OxCoder-9B — 9,409,813,744 parameters (BF16), a 9B coding model (32-layer dense Qwen3.5 hybrid: 24 Gated DeltaNet linear-attention and 8 full-attention layers, every 4th being full attention; hidden 4096, FFN 12288, 16 attention / 4 KV heads, 262,144-token context), text + image — built and measured on an AMD Ryzen AI Max+ 395 (Radeon 8060S, gfx1151). The checkpoint ships no mtp.* tensors (it declares mtp_num_hidden_layers: 1); every file here carries a grafted in-file MTP head from the declared base model.

• Grafted MTP head inside every file. Measured on ROCm0 at n-max 3: 64.36 tok/s vs 38.50 without it (1.67×), draft acceptance 0.743 (Speed).
• Prompt caching with MTP needs the included server patch in recipe/patches/ — without it, a build of this ROCmFPX line reuses nothing while the head is loaded (measured).
• Vision projector included.
• Importance-matrix builds of the same three 4-bit tiers: kingjones777/OxCoder-9B-MTP-ROCmFP4-imatrix-GGUF.

Which file should I use?

AMD Ryzen AI Max+ 395 (Radeon 8060S) (MAX-1), ROCm 7.2.4, patched llama-server (see Quick start), -c 65536, one request at a time (--parallel 1), greedy (temp 0, top_k 1), ignore_eos so every arm generates exactly 256 tokens after a code prompt of 7,094–7,102 tokens (the first 30,000 characters of convert_hf_to_gguf.py plus an instruction), a unique nonce per request and cache_prompt: false (cache_n = 0 asserted on every timed request), 1 warm-up then the median of 3. Decode numbers are the server's own predicted_per_second. Box iced: no other model loaded (except the rows marked †, see Vulkan0 prefill and CPU activity).

TG = decode tokens/s after the 7,094–7,102-token code prompt, grafted MTP head, --spec-draft-n-max 3. PP = prefill tokens/s on ROCm0. ¹ The BF16 PPL shown is the paired base every "×" ratio is computed against (averaged over the same scored tokens in the KL-divergence runs). The standalone BF16 run's own summary line reads 6.6532 ± 0.0813. ² Quality columns: see Where the quality numbers come from below. ³ BF16 conversion of the grafted checkpoint (MTP block included); not published. ⁴ BPW as printed by llama-quantize: bits per weight over the 9,197,093,888 weights in each GGUF — the checkpoint's 8,953,803,264 text weights plus the grafted 243,290,624-weight MTP block. The 9,409,813,744-parameter count above is the checkpoint's: it includes the 456,010,480-weight vision tower, which ships in the projector file, and no MTP block. † Quiet re-run: the first measurement had requests inside a window when an unrelated job ran on this box; re-run later with other model servers loaded but idle — see Vulkan0 prefill and CPU activity. ‡ Has requests inside a window when an unrelated job ran on this box; kept as measured — see Vulkan0 prefill and CPU activity.

Start with STRIX_LEAN. COHERENT's KLD is within noise of it (-2.4 %, 0.4σ) for 315 MiB more; STRIX_LEAN decodes 14.6 % faster than COHERENT on ROCm0 (65.80 vs 57.40† tok/s; per-run ranges 64.36–65.83 and 55.67–58.10) and within 0.7 % of it on Vulkan0 (60.98‡ vs 60.53‡), and prefills 11.8 % faster than it on ROCm0 (897.8 vs 803.4† tok/s).

FAST does not buy a clear speed gain here: FAST decodes within 1.5 % of STRIX_LEAN on ROCm0 (64.81† vs 65.80 tok/s; per-run ranges 62.61–68.03 and 64.36–65.83) and within 1.2 % of it on Vulkan0 (61.72 vs 60.98‡), and prefills within 0.1 % of it on ROCm0 (898.6† vs 897.8 tok/s); its KLD is within noise of STRIX_LEAN's (0.0890 vs 0.0879, +1.3 %, 0.2σ).

KLD order (lower is closer to BF16; KLD gaps under 2σ are called within noise): COHERENT 0.0857, STRIX_LEAN 0.0879 (+2.5 % vs COHERENT), FAST 0.0890 (+3.8 % vs COHERENT). STRIX_LEAN is 315 MiB smaller than COHERENT; FAST is 178 MiB smaller than STRIX_LEAN. Speed gaps are called a tie below 10.3 % for decode and below 3.0 % (ROCm0) / 3.0 % (Vulkan0) for prefill (3.0 % when the two backends are compared): the larger of 3 % and the widest gap measured between two files that run identical tensors — each imatrix file and its standard twin at the MTP knee (decode 10.3 %; prefill 1.3 % on ROCm0, 0.2 % on Vulkan0). The decode floor is the widest because each twin's drafts are accepted at a different rate (STRIX_LEAN on ROCm0: 0.750 standard vs 0.641 imatrix): the imatrix changes the main model's weights, so the head sees different hidden states and is checked against different target tokens.

The imatrix build is measurably closer to BF16 at the same size: STRIX_LEAN 0.0879 → 0.0610 (-30.6 %, 6.2σ); COHERENT 0.0857 → 0.0550 (-35.9 %, 7.4σ); FAST 0.0890 → 0.0637 (-28.4 %, 5.8σ).

Quality is graded against the BF16 GGUF (reference logits computed on CPU) on a held-out corpus (wikitext-2 test, -c 2048, 40 chunks × 1,023 scored tokens each — the second half of every window, less its first token — = 40,920), never on the imatrix calibration text. KLD is the per-token KL divergence of each quant's next-token distribution from BF16's on the same tokens — far more sensitive than perplexity.

Where the quality numbers come from. Measured directly on these files, against BF16 logits computed on CPU in the same session. The BF16 model's own first-window perplexity was CPU 6.6290, Vulkan0 6.6346 (+0.08 %), ROCm0 6.6463 (+0.26 %): the two GPU backends disagree with CPU and with each other, so the reference logits come from pure CPU (-dev none -ngl 0 --no-op-offload), and that run's own first-window value (6.6290) equals the CPU control's. Every file was graded on ROCm0 (the columns above); the standard STRIX_LEAN file was also graded on Vulkan0. The standard STRIX_LEAN grade was run twice on ROCm0 and all 40 per-chunk rows matched.

Same file, same reference, graded on each backend — standard STRIX_LEAN KLD ROCm0 / Vulkan0: 0.0879 / 0.0886 (+0.9 %, 0.2σ). The two backends agree within noise on this file.

Quick start

1. Download

hf download kingjones777/OxCoder-9B-MTP-ROCmFP4-GGUF --local-dir ~/models/oxcoder

2. Build llama-server with the prompt-cache patch — ROCmFPX at the measured commit (ROCm and Vulkan prerequisites: the project's build guide)

git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
git checkout d3ca53726109b46fd5cd6fa42f688c707079b44e
git apply ~/models/oxcoder/recipe/patches/mtp-prompt-cache-fix.patch
HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \
  -DGGML_HIP=ON -DGGML_VULKAN=ON -DGPU_TARGETS=gfx1151 \
  -DGGML_HIP_GRAPHS=ON -DGGML_HIP_NO_VMM=ON -DLLAMA_CURL=OFF
cmake --build build --target llama-server -j

(The CMake options of the measured build are listed in Reproduction.)

d3ca537 is also in the history of the official ROCmFPX/ROCmFPX repository; the patch was verified against d3ca53726109b46fd5cd6fa42f688c707079b44e only.

3. Serve — the MTP head is inside the model file, so there is no --model-draft:

Text and tool calls, with MTP:

env LD_LIBRARY_PATH=$PWD/build/bin:/opt/rocm/lib HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
build/bin/llama-server \
  -m ~/models/oxcoder/OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 --spec-draft-n-min 0 --spec-draft-p-min 0.0 \
  -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -dev ROCm0 \
  -c 65536 --host 127.0.0.1 --port 8080

Images (--mmproj), without MTP:

env LD_LIBRARY_PATH=$PWD/build/bin:/opt/rocm/lib HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
build/bin/llama-server \
  -m ~/models/oxcoder/OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf \
  --mmproj ~/models/oxcoder/mmproj-OxCoder-9B-BF16.gguf \
  -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -dev ROCm0 \
  -c 65536 --host 127.0.0.1 --port 8080

⛔ Images: with the MTP head loaded, image requests fail on this build (measured). For image input, drop the two --spec-* lines.

(LD_LIBRARY_PATH avoids a soname clash on machines that also have a Vulkan-only llama.cpp build.) The exact measured argv is in Reproduction.

4. Call — thinking off (OxCoder's SWE-bench sampling: temperature 1.0, top_p 0.95):

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "messages": [{"role": "user", "content": "Hello"}],
    "temperature": 1.0,
    "top_p": 0.95,
    "chat_template_kwargs": {"enable_thinking": false}
  }'

⚠️ On an unpatched build the in-file head still drafts, but prompt caching turns off while it is loaded — every turn re-prefills the whole conversation (measured). For multi-turn / agentic use there, drop the two --spec-* lines.

Requires a llama.cpp build with ROCmFP4 / ROCmFPX tensor-type support; stock llama.cpp rejects these tensor types.

Reasoning controls

OxCoder ships its own chat template (template_version qwen3.6-froggeric-v21.3, in the source repo's chat_template.jinja). It reads these chat_template_kwargs:

A <|think_off|> or <|think_on|> tag inside a system or user message overrides enable_thinking (the last tag wins) and is removed from the text.

A short tool result that starts like an error (error:, Traceback, fatal:, command not found …) gets a "⚠️ SYSTEM WARNING" appended by the template; after two such results in a row (no user turn in between), the next generation prompt also gets an empty think block, regardless of enable_thinking. A developer message is treated as system; any other unknown role is rendered as a user turn [role]: content.

With thinking on and a small max_tokens, the whole budget can go to reasoning and content comes back empty — raise max_tokens before concluding the model is broken.

Tool calls in the default format are parsed by llama.cpp natively through the Jinja chat template (on by default). OxCoder's own card reports SWE-bench at temperature 1.0 / top_p 0.95 and Terminal-Bench / NL2Repo at temperature 1.0 / top_p 1.0. The tools gate on this card used temperature 1.0, top_p 0.95, top_k disabled, max_tokens 8192.

Speed: making it fast

Measured on OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf (patched server; 7,094–7,102-token code prompt):

On ROCm0, n-max 3 is the fastest depth of the n-max 1–6 sweep: 64.36 tok/s, 1.67× the 38.50 tok/s no-draft baseline, with 0.743 of drafted tokens accepted. On Vulkan0 (n-max 2–4 measured, around the ROCm0 knee) the fastest is n-max 3: 61.95 tok/s, 1.60× its own 38.65 tok/s no-draft baseline, acceptance 0.684 — within 3.7 % of ROCm0 at its knee, a tie. The separate DFlash drafter (z-lab/Qwen3.5-9B-DFlash, Q8_0; n-max 4, 8, 15 measured on ROCm0) peaks at n-max 4: 48.99 tok/s, 1.27× the no-draft baseline, acceptance 0.615 — 23.9 % slower than the grafted MTP head at its knee. Prefill with the head loaded vs without it, same prompt: ROCm0 897.8 vs 1089.2 tok/s; Vulkan0 614.2‡ vs 668.3 tok/s. Speed gaps are called a tie below 10.3 % for decode and below 3.0 % (ROCm0) / 3.0 % (Vulkan0) for prefill (3.0 % when the two backends are compared): the larger of 3 % and the widest gap measured between two files that run identical tensors — each imatrix file and its standard twin at the MTP knee (decode 10.3 %; prefill 1.3 % on ROCm0, 0.2 % on Vulkan0). The decode floor is the widest because each twin's drafts are accepted at a different rate (STRIX_LEAN on ROCm0: 0.750 standard vs 0.641 imatrix): the imatrix changes the main model's weights, so the head sees different hidden states and is checked against different target tokens.

Draft acceptance depends on how predictable the text is, so the same file was also run on prose. STRIX_LEAN, code prompt (7,095–7,101 tokens: the first 30,000 characters of convert_hf_to_gguf.py plus a coding instruction) vs prose prompt (7,450–7,458 tokens: the first 34,000 characters of wikitext-2 train plus a writing instruction) — ROCm0 with the head: code 65.80 tok/s vs prose 48.00 (prose decodes 27.1 % slower than code); Vulkan0 with the head: code 60.98‡ tok/s vs prose 49.75 (prose decodes 18.4 % slower than code); ROCm0 without the head: code 38.50 tok/s vs prose 38.46 (prose decodes within 0.1 % of code); Vulkan0 without the head: code 38.65 tok/s vs prose 38.55 (prose decodes within 0.3 % of code). (Same tie rule as above.)

Every 4-bit file at the MTP knee (n-max 3), both backends — separate runs from the depth sweep above (STRIX_LEAN at n-max 3 read 64.36 tok/s on ROCm0 and 61.95 on Vulkan0 there):

STRIX_LEAN: ROCm0 decodes within 7.9 % of Vulkan0 (65.80 vs 60.98‡ tok/s; ranges 64.36–65.83 / 60.86–61.24) and prefills 46.2 % faster than it (897.8 vs 614.2‡ tok/s). COHERENT: ROCm0 decodes within 5.2 % of Vulkan0 (57.40† vs 60.53‡ tok/s; ranges 55.67–58.10 / 58.45–60.74) and prefills 13.3 % faster than it (803.4† vs 709.3‡ tok/s; all three Vulkan0 runs overlapped the unrelated job — a quiet re-run read 621.6 tok/s prefill). FAST: ROCm0 decodes within 5.0 % of Vulkan0 (64.81† vs 61.72 tok/s; ranges 62.61–68.03 / 61.72–61.81) and prefills 45.4 % faster than it (898.6† vs 618.0 tok/s). (Same tie rule as above.)

Vulkan0 prefill and CPU activity

While the per-file rows were being measured, an unrelated job ran on this box from 01:50:50 to 01:53:40 UTC: one niced, single-threaded process re-rendering another model's cards, no GPU use (recipe/logs/O6_background_job_journal.txt). Rows with requests inside that window, each against the closest like-for-like requests outside it: STRIX_LEAN on Vulkan0, run 3: 718 tok/s prefill vs 610–614 for runs 1 and 2 of the same row; COHERENT on Vulkan0, all three runs: 709 tok/s prefill vs 604–609 for its imatrix twin (identical tensor types, outside the window); COHERENT on ROCm0, all three runs: 809–811 tok/s prefill vs 810–815 for its imatrix twin (identical tensor types, outside the window); FAST on ROCm0, run 1: 910 tok/s prefill vs 906–910 for runs 2 and 3 of the same row.

Re-run with the same harness and flags, quiet vs with one niced busy-loop thread (whole-box CPU busy 1.2–3.6 % quiet, 4.4–6.7 % with the thread; this re-run was not on an iced box: 5 other model servers were loaded, all idle before and after every arm):

On this box a single busy CPU thread made Vulkan0 prefill with the MTP head 14–15 % faster; ROCm0 prefill moved -0.2 %. Vulkan0 prefill numbers therefore depend on what else the CPU is doing. A second quiet COHERENT Vulkan0 run after the busy-thread arms read 620 tok/s (first quiet run: 622). The ROCm0 quiet control (imatrix COHERENT, from an undisturbed window) reproduced its original row (prefill 812.9 → 807.4 tok/s, -0.7 %; decode +0.6 %; tolerances ±3 % / ±5 %, set before the re-run), so the COHERENT and FAST ROCm0 rows show the quiet re-run (marked †). The Vulkan0 quiet control (imatrix STRIX_LEAN, from an undisturbed window) did not reproduce its original row within the tolerances set before the re-run (prefill 612.9 → 631.6 tok/s, +3.05 % against ±3 %; decode +2.0 % against ±5 %; the re-run's box state differed: other model servers were loaded), so the STRIX_LEAN and COHERENT Vulkan0 rows keep the original values (marked ‡): STRIX_LEAN's reported medians are values from its runs outside the window (its in-window run 3 is the extreme of the three in both prefill and decode); all three COHERENT runs were inside the window, so its reported values were measured with the job running (its quiet re-run read 622 tok/s prefill and 59.36 tok/s decode). The replaced rows stay in recipe/results_summary.json under bench_superseded.

The grafted MTP head

The OxCoder checkpoint has no mtp.* tensors. The 15 mtp.* tensors of Qwen/Qwen3.5-9B (the declared base_model) were fetched by HTTP range read and written byte-for-byte as an extra safetensors shard next to the untouched OxCoder file; the converter keeps mtp.* as the in-file nextn block. The head was trained for the base model, not for OxCoder.

At n-max 3 on ROCm0, the grafted head's measured acceptance was 0.743 and STRIX_LEAN decoded at 64.36 tok/s — 1.67× the 38.50 tok/s no-draft baseline on the same backend and the same prompt. The head was trained for Qwen3.5-9B, not for OxCoder; those figures are measurements on the benchmark prompts above, not a claim about other prompts.

Is the output the same as without MTP?

Greedy output with MTP matched greedy output without a draft head on 2 of 6 prompts (every pair run cold at temperature 0). The 6 prompts are the identity gate's fixed prompt (1) plus the 5 prompts the aligned cache gate also ran without a draft head (o-c3-* rows; cold reply compared with cold reply). Without a draft head, the unpatched and patched servers produced identical output on the fixed prompt. A mismatch is not a quality regression: with a draft head loaded the server evaluates the same math in different batch shapes (drafts are verified several tokens at a time, and the head needs the hidden state of every prompt position), so float rounding can differ and a near-tie can break the other way; from that token on, the two replies diverge. If you need output that is byte-for-byte reproducible against a no-MTP run, serve without --spec-type.

Prompt caching with MTP

Measured: a second turn that shares the first turn's long prefix and differs only in its closing instruction, each exchange run warm (cache_prompt: true) and then cold (cache_prompt: false) and the two replies compared byte for byte. Every prompt in the patched gate is padded to one token length so warm and cold see identical chunking.

⛔ Unpatched, loading the draft head turns prompt reuse off. On the unmodified d3ca537 server with the grafted head loaded, a second turn sharing a long prefix reused 0 of its 7,067 prompt tokens.

✅ Patched, reuse works. In 5 two-turn exchanges at a fixed prompt length of 7,039 tokens, every second turn resumed from the checkpoint 1,028 tokens before its end — 6,011 tokens reused (85 %), 1,028 processed — median prefill 1.3 s instead of 7.8 s cold (6.2× faster), with the draft head still loaded.

Resuming does not change the output: each warm reply was byte-identical to a cold run of the same prompt in 5/5 exchanges with the grafted head. Patched with no draft head (control): warm = cold in 5/5 exchanges.

One behaviour to know about, which belongs to llama-server rather than to this patch: the server processes the last n_ubatch + 4 tokens of every prompt as two separate batches so it can checkpoint there (1,024 + 4 = 1,028 tokens with the -ub 1024 used in these measurements, where the server default is -ub 512; upstream PR #20288). A turn that resumes from a checkpoint left by a prompt of a different length therefore splits its tail differently from a cold run, and float rounding can flip a greedy token. Both replies are ordinary greedy decodes; they are just not bit-identical to each other. That is why every prompt in this gate has the same length.

Tool calling

The template emits the XML-style <tool_call><function=…><parameter=…> format, which llama.cpp parses natively through the Jinja chat template (on by default). Suite run through llama-server in the deployed configuration (grafted MTP on):

14/14 on native tool_calls (raw XML left in content counts as a failure), run on OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf with the grafted head on, at OxCoder's SWE-bench sampling (temperature 1.0, top_p 0.95, top_k disabled, max_tokens 8192).

Every check passed in both thinking modes (enable_thinking: true and false).

Vision

mmproj-OxCoder-9B-BF16.gguf is the 27-layer vision tower (width 1152), loaded with --mmproj. Its attention follows the server's -fa setting, so both settings were checked, with and without the MTP head.

Probe: a synthetic 448×448 image with a red circle and a blue square (a model that ignores the image cannot name both), sent to OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf on the patched server with --mmproj, temperature 0. Pass = the reply names every expected term (red, blue, circle, square).

⛔ With the MTP head loaded, image requests fail with -fa on and with -fa off — the server aborted (process: missing MTP boundary for seq_id=0 pos=17 (current=3/1 previous=2/1) → update_slots: failed to process speculative batch). Without the head they work. For image input, serve without --spec-type. The unmodified d3ca537 server aborts the same way, so this is not caused by the prompt-cache patch; the server log points to the MTP state not being advanced across an image's positions, which jump (M-RoPE).

Reply (no head, -fa on; first 300 characters as stored by the harness):

The image contains two distinct geometric shapes placed side by side on a white background:

1. Left Shape: A solid red circle. It is perfectly round with no visible outline or gradient — just a flat, uniform red fill.

2. Right Shape: A solid blue square. It has sharp, straight edges and ri

Memory

Measured footprint (drop in MemAvailable) of STRIX_LEAN on ROCm0 with the grafted MTP head loaded (n-max 3, --spec-draft-p-min 0.0), the vision projector, q8_0 KV cache, --no-mmap, -cram 512, one slot, patched server. After one request is after a single request whose prompt is the first 30,000 characters of convert_hf_to_gguf.py plus a one-line instruction, with 64 tokens generated.

No row was decode-benchmarked beyond that one request. Nothing beyond these rows was measured.

Quantization methodology

# 0. graft the 15 mtp.* tensors of Qwen/Qwen3.5-9B onto the OxCoder checkpoint (byte-for-byte extra shard):
#    hf/ is the untouched OxCoder download, graft/ a hard-linked copy of it plus the new shard
cp -al hf graft && rm -rf graft/.cache
python3 oxcoder_graft.py hf graft/model-mtp-graft.safetensors

# 1. convert: grafted checkpoint WITH its MTP layer (the converter's default), and the vision projector
python convert_hf_to_gguf.py graft --outtype bf16 --model-name OxCoder-9B --outfile OxCoder-9B-MTP-BF16.gguf
python convert_hf_to_gguf.py graft --outtype bf16 --mmproj --model-name OxCoder-9B --outfile mmproj-OxCoder-9B-BF16.gguf

# 2. quantize from BF16 only; the LM head is forced up on every tier and read back by exact tensor name
llama-quantize --output-tensor-type q6_K                            OxCoder-9B-MTP-BF16.gguf OUT Q4_0_ROCMFP4_STRIX_LEAN 8
llama-quantize --output-tensor-type q6_K --token-embedding-type q6_K OxCoder-9B-MTP-BF16.gguf OUT Q4_0_ROCMFP4_COHERENT 8
llama-quantize --output-tensor-type q6_K                            OxCoder-9B-MTP-BF16.gguf OUT Q4_0_ROCMFP4_FAST 8

# 3. BF16 reference logits on CPU (chunk-1 control, BF16 first-window PPL: CPU 6.6290, Vulkan0 6.6346, ROCm0 6.6463)
llama-perplexity -m OxCoder-9B-MTP-BF16.gguf -f wikitext-2-raw/wiki.test.raw -c 2048 -b 2048 --chunks 40 --kl-divergence-base bf16.kld \
  -dev none -ngl 0 --no-op-offload -t 16
# 4. grade each shipped file against those logits on ROCm0 (STRIX_LEAN also on Vulkan0)
llama-perplexity -m OUT --kl-divergence-base bf16.kld --kl-divergence -c 2048 -b 2048 -ngl 999 -fa on -dio -dev ROCm0
llama-perplexity -m OUT --kl-divergence-base bf16.kld --kl-divergence -c 2048 -b 2048 -ngl 999 -fa on -dio -dev Vulkan0

# 5. DFlash drafter (not redistributed): this fork's converter, then Q8_0
python convert_hf_to_gguf.py dflash-hf --outtype bf16 --target-model-dir graft \
  --outfile gguf/dflash-Qwen3.5-9B-BF16.gguf
llama-quantize gguf/dflash-Qwen3.5-9B-BF16.gguf out/dflash-Qwen3.5-9B-Q8_0.gguf Q8_0 16

Receipts (the built file is the receipt — exact tensor names, never a substring match; recipe/logs/):

Where the three tiers differ, read from the standard files (blk.N = every layer that has the tensor, the MTP block included; families not listed have the same type in all three):

tie_word_embeddings is false, so the output head is a real tensor and --output-tensor-type q6_K does real work. All three tiers pin output.weight to q6_K; COHERENT also pins token_embd.weight to q6_K, while STRIX_LEAN and FAST keep their tier's own embedding type (shown in the receipts).

Reproduction

server   : https://github.com/charlie12345/ROCmFPX @ d3ca53726109b46fd5cd6fa42f688c707079b44e
           + recipe/patches/mtp-prompt-cache-fix.patch (sha256 412a0173f60faa937018f0153efe9b5d9230c1606735502485e7e520fffb68dd)
           built target llama-server with the same CMake options as below; sha256 14e9f1299240c891478c820b8c20811ca5721044d2e63e23799eaa9be26b30bd
tools    : same commit, unpatched (quantize / imatrix / perplexity; the patch touches only common/ and tools/server/)
           build dir /opt/llama-rocm/rocmfpx-724/build-hipvk, Release, Unix Makefiles, GGML_HIP=ON GGML_VULKAN=ON
           GGML_HIP_GRAPHS=ON GGML_HIP_NO_VMM=ON GGML_NATIVE=ON AMDGPU_TARGETS=gfx1151 LLAMA_CURL=OFF
           (this CMake forwards AMDGPU_TARGETS to GPU_TARGETS, the spelling used in Quick start)
           CMAKE_HIP_COMPILER=/opt/rocm-7.2.4/lib/llvm/bin/clang
           sha256 llama-quantize   1559189968a30781f837e4d25ca3cdf26c8eb0351023675406535eddcbe0db13
           sha256 llama-imatrix    bfdce5c21a01bdc8043d69335b74a7b68b8335ae989991a1e10e21d5faeaeb8e
           sha256 llama-perplexity c2175925bb0c61df89ff787845d8e03a35403f684be52719ac2987b8ca636a53
           sha256 llama-server     d077b98f3f04df4b482a5826c8922e39cda3aa8d2e2b2fbe1870f36d245cebe7   (unpatched; used for the unpatched control rows: prompt cache and vision)
source   : OrionLLM/OxCoder-9B  revision 07ba04e2149bace1b43c66448471a70378a7a708
model    : OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf (the argv below; every file was measured the same way)
argv     : /opt/llama-rocm/rocmfpx-724-mtpcache/build-hipvk/bin/llama-server -m /mnt/models/oxcoder-9b/out/OxCoder-9B-MTP-Q4_0_ROCMFP4_STRIX_LEAN.gguf -dev ROCm0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-n-min 0 --spec-draft-p-min 0.0
env      : LD_LIBRARY_PATH=<patched build>/bin:/opt/rocm-7.2.4/lib
           HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1
box      : aimax — AMD RYZEN AI MAX+ 395 w/ Radeon 8060S (gfx1151), 124 GiB RAM, GTT 131072 MiB,
           kernel 6.17.6-061706-generic, ROCm 7.2.4
protocol : 7,094–7,102-token code prompt, 256 generated tokens, temp 0 / top_k 1, ignore_eos, cache_prompt false,
           1 warm-up + median of 3, no co-resident models (box iced)
           (rows marked † were re-run later, quiet, with other model servers loaded but idle)
measured : 2026-09-16 to 2026-09-17, by the pipeline in recipe/ (every raw number in recipe/results_summary.json and recipe/raw/)

Files

The DFlash drafter (dflash-Qwen3.5-9B-Q8_0.gguf, from z-lab/Qwen3.5-9B-DFlash @ 5fc3b3d474760f18c516db87d84c37edbfd3ede6) is not redistributed. The methodology block shows how to convert it with this fork (--target-model-dir) and quantize Q8_0.

SHA256SUMS covers every model file. recipe/ holds the measurement pipeline (recipe/pipeline/), the prompt-cache patch (recipe/patches/), the graft provenance (recipe/graft/), raw per-run results (recipe/raw/), build and receipt logs (recipe/logs/), and results_summary.json with every measured value on this card. Architecture facts (layer counts, vocabulary, vision depth) come from the checkpoint's config.json at revision 07ba04e2149bace1b43c66448471a70378a7a708.

Known issues and limits

• The MTP head is grafted from the base model. The 15 mtp.* tensors come from Qwen/Qwen3.5-9B at revision c20223623576 (recipe/graft/GRAFT_PROVENANCE.json). They were trained for that checkpoint, not for OxCoder's fine-tune; their usefulness here is the measured acceptance and speed on this box, not a task-accuracy claim.
• MTP + image requests fail on this build (they work with the head unloaded) — see Vision. For image input, drop the two --spec-* lines.
• Prompt caching with MTP needs the included server patch. On the unpatched server the in-file head still drafts (40 of 62 drafted tokens accepted in that control), but the measured second turn reused 0 of its 7,067 prompt tokens; for multi-turn use on an unpatched build, drop the two --spec-* lines.
• preserve_thinking defaults to true in this chat template: earlier assistant turns are re-rendered with their reasoning, so multi-turn contexts grow faster than with templates that drop it. Pass "preserve_thinking": false in chat_template_kwargs to drop it.
• Tool calling was measured only with the template's default tool_call_format (XML parameters); the JSON variant ("tool_call_format": "json") was not tested.
• llama-server's host-RAM prompt cache defaults to 8 GiB (-cram 8192). On a shared box, set -cram explicitly.
• Measured on Linux only (AMD RYZEN AI MAX+ 395 w/ Radeon 8060S, ROCm 7.2.4, patched d3ca537).
• Not measured: decode beyond a 7,458-token prompt (a 262,144-token context was loaded in the memory test, not benchmarked), long-context quality, video input, concurrency above 1, and task-level accuracy. Perplexity/KLD measure next-token fidelity to BF16 on prose, not reasoning or code correctness. OxCoder's own SWE-bench / Terminal-Bench / NL2Repo numbers are for the unquantized checkpoint, not these files.

License and attribution

Apache-2.0, inherited from the base model. Weights: OrionLLM (OrionLLM/OxCoder-9B). MTP head weights: Qwen (Qwen/Qwen3.5-9B, Apache-2.0). DFlash drafter: z-lab (not redistributed). ROCmFP4 / ROCmFPX quantization format and runtime: the ROCmFPX project. Patch, graft, quantization and measurements: kingjones777.