Agents-A1-Abliterated-MTP — ROCmFP4 for AMD Strix Halo (gfx1151)

The model here is not our work. The abliteration is huihui-ai's (Huihui-Agents-A1-abliterated-GGUF); the MTP block was extracted from SC117's (Agents-A1-Uncensored-MTP-APEX-GGUF); the base model is InternScience's (Agents-A1). This repository adds only the MTP graft, the ROCmFP4/ROCmFPX quantisation ladder, the dual-backend build, and the measurements. Go star their repos.

Four quantisations of the abliterated Agents-A1 35B-A3B MoE, each carrying a working Multi-Token-Prediction block, built for AMD Ryzen AI Max+ 395 / Radeon 8060S. Every file runs on BOTH the HIP (ROCm) and Vulkan backends from a single binary — the backend is a runtime -dev flag, not a rebuild.

Lead result: the released InternScience/Agents-A1 checkpoint ships no MTP layer, so no abliterated build of this model could speculate at all. We grafted one in at BF16 before quantising. It accepts at 0.963 (mean accepted length 3.44 of 4) and is worth +19.0 % on Vulkan and +20.0 % on ROCm.

⛔ Read the serving note before you deploy this. With thinking enabled this model will return an empty content field on most non-code prompts — it never stops reasoning inside a sane token budget. The upstream non-abliterated checkpoint does exactly the same thing. Set enable_thinking: false. Details and the control-arm measurement are below.

---

Which file should I use?

mmproj-Agents-A1-Abliterated-BF16.gguf (0.84 GiB) is the vision projector, inherited unchanged from the source repo — it was not abliterated and not quantised. Pass it with --mmproj for image input, and use -fa off for vision scoring.

STRIX_LEAN (106) is the tier to take — it is both the smallest and the fastest here, at every workload on both backends. That is unusual and we did not expect it.

⛔ ftype 105 (STRIX) is deliberately not published — it leaves output.weight and the *_shexp tensors at 4 bit.

---

Quick start — the backend is one flag

# Vulkan — best decode (chat, agentic loops)
llama-server -m Agents-A1-Uncensored-MTP-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  -dev Vulkan0 -ngl 999 -fa on -fit off -np 1 -c 8192 --jinja \
  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0 \
  --host 0.0.0.0 --port 8080

# HIP / ROCm — best prefill (long prompts, cold starts, RAG)
#   SAME FILE. Only -dev changes.
llama-server -m Agents-A1-Uncensored-MTP-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  -dev ROCm0 -ngl 999 -fa on -fit off -np 1 -c 8192 --jinja \
  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0 \
  --host 0.0.0.0 --port 8080

Send requests with thinking disabled:

{"messages":[{"role":"user","content":"..."}],
 "max_tokens":1600,
 "chat_template_kwargs":{"enable_thinking":false}}

These MTP flags are for single-shot use. For multi-turn / agentic work drop them entirely — see Multi-turn agentic work below, where speculation costs 31 % end-to-end. For single-shot, --spec-draft-p-min 0.0 is not optional if you want the speed. The default is 0.75 and it throttles the draft — see the sweep below.

⛔ You need a ROCmFPX build with Vulkan enabled — stock llama.cpp will NOT load these files

ROCmFP4/ROCmFPX use ggml tensor types 100–119. Upstream llama.cpp's type table stops at 43, so a stock build rejects these files outright, on any backend:

gguf_init_from_reader: tensor '...' has invalid ggml type 101. should be in [0, 43)

Build one with both backends — this is the exact configuration behind every number on this card:

git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
cmake -B build-hipvk -G "Unix Makefiles" \
  -DCMAKE_MAKE_PROGRAM=/usr/bin/gmake \
  -DCMAKE_C_COMPILER=/usr/bin/cc -DCMAKE_CXX_COMPILER=/usr/bin/c++ \
  -DCMAKE_HIP_COMPILER=/opt/rocm-7.2.4/lib/llvm/bin/clang \
  -DROCM_PATH=/opt/rocm-7.2.4 -DCMAKE_PREFIX_PATH=/opt/rocm-7.2.4 \
  -DGGML_HIP=ON -DGGML_VULKAN=ON -DAMDGPU_TARGETS=gfx1151 \
  -DVulkan_GLSLC_EXECUTABLE=/usr/bin/glslc \
  -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF -DLLAMA_BUILD_WEBUI=OFF
cmake --build build-hipvk -j12
build-hipvk/bin/llama-server --list-devices   # must list ROCm0 AND Vulkan0

Build traps, all hit for real: no Ninja on this box — use Unix Makefiles + gmake, and pass -DCMAKE_C_COMPILER/-DCMAKE_CXX_COMPILER explicitly or the HIP language probe dies with CMAKE_C_COMPILER not set, after EnableLanguage; the HIP compiler is at /opt/rocm-7.2.4/lib/llvm/bin/clang, not /opt/rocm/llvm/bin/clang++; glslc is not auto-detected.

---

The MTP graft — what we actually did

InternScience/Agents-A1 is a 40-layer Qwen3.5-35B-A3B MoE with no MTP layer, and the abliteration inherits that: 733 tensors, no blk.40. Speculative decoding is unavailable.

We spliced one in at BF16, before quantisation:

recipient   abliterated BF16      733 tensors   69,376,636,896 bytes
donor       SC117 MTP BF16        blk.40.*       20 tensors / 1,690,355,712 bytes
result      abliterated+MTP BF16  753 tensors   71,066,993,952 bytes
metadata    qwen35moe.block_count 40 -> 41 · qwen35moe.nextn_predict_layers absent -> 1

The recipient's tensor data ends at relative offset 69,365,647,872 — byte-for-byte where the donor's blk.40 begins. The two files share an identical blk.0–blk.39 layout, so the graft is a header rebuild plus an append, not a re-encode: every abliterated weight is bit-identical to the source.

The MTP block carries no shared_head_head or embed_tokens of its own — it ties to the model's own token_embd/output, so the grafted head reads through the abliterated vocabulary projection.

We expected this to accept badly and it did not. The donor MTP is trained against base Qwen3.5-35B-A3B; the recipient is an abliterated Agents-A1 — two different post-trainings. It accepts at 0.963, mean accepted length 3.35–3.46 of 4 across all four tiers.

---

Measured performance

AMD Ryzen AI Max+ 395 / Radeon 8060S (gfx1151), 128 GB unified, GTT 131072 MiB, ROCm 7.2.4, Mesa 25.2.8 (RADV), kernel 6.17.6, Ubuntu 24.04.4, governor performance, box idle — 0 co-resident seats. -c 4096 -fa on -fit off -np 1, temperature 0, cache_prompt:false, unique nonce per request, 400 tokens, 1 warm-up discarded, median of 12.

All four tiers × both backends (MTP on, p_min 0.0)

⚠️ The prefill figures in the table above were measured on ~30-token prompts and understate real performance badly. On realistic long prompts (1,800–7,600 tokens) this model prefills at 745–813 tok/s. Short-prompt pp is not a meaningful number — quote long-prompt prefill.

What MTP is worth — STRIX_LEAN, single-shot, speculation off vs on

--spec-draft-p-min sweep — STRIX_LEAN, MTP on, n_max 4

Tune on tok/s, not on acceptance. p_min 0.25 and 0.5 return a higher mean accepted length than 0.0 and are still slower. The headline acceptance rate is not the objective function.

In the recommended serving mode (enable_thinking: false, 1600 tokens)

The findings that matter

1. MTP is worth having on this architecture — which is not a given. We have measured MTP as a net loss (0.87–0.94×) on a different qwen35moe 35B-A3B model on this same hardware and fork. Here it is +19–20 %. Do not assume either result transfers; measure it.
2. The backend crossover flips by tier. Vulkan wins decode on the 4-bit tiers; on Q6_LEAN ROCm wins outright (65.91 vs 61.27) and uses 7.2 GiB less GTT. ROCm keeps the prefill crown everywhere (478 vs 260 on STRIX_LEAN, ~+84 %).
3. The p_min default costs real throughput — +4.6 % Vulkan, +5.8 % ROCm just by opening it.
4. The smallest tier is the fastest tier. STRIX_LEAN (17.88 GiB) beats COHERENT (18.93 GiB) on every workload and both backends.

⚠️ These rankings are model-specific and we say so. Measure them on your own model.

---

Silent-truncation gate — 480 generations, 0 truncated

On this hardware a Vulkan failure can be silent: 1 token, empty content, clean exit, /health still 200. A benchmark reporting only a median cannot see it. We count generations that actually reach n_predict:

Speculative decoding works on Vulkan on this model. We have separately measured MTP failing on Vulkan on gfx1151 for a different model; it does not fail here, across 240 Vulkan generations.

---

⛔ Serving note — this model does not stop thinking

This is the single most important thing on this page.

With thinking enabled, at max_tokens = 1600, non-empty content returned in 6 reps per cell:

Raising the budget does not fix it — reasoning_content simply grows (6013 chars on prose at 1600) and content never arrives. The code column is noisy at n=6 and not monotonic; we report it as measured rather than smoothing it.

This is upstream behaviour, not damage from the abliteration or the graft. We ran the non-abliterated InternScience/Agents-A1-Q4_K_M as a control on the same three workloads, same 1600-token budget, 4 reps:

With enable_thinking: false both are clean, and this build finishes on stop for all three workloads where the control still hits length on prose.

Stated with its denominator: with thinking on at 1600 the control produced 650 chars on prose and this build produced 0 (n = 4). Both are budget-starved and terminate on length; the difference is whether the closing think tag lands inside the window. We report it because it is in the data.

---

---

⛔ Multi-turn agentic work — turn MTP OFF

This is the most important correction on this card. The speculative-decoding flags that make single-shot generation fast break the prompt cache on multi-turn work. The server reports:

prompt cache cold fallback: reason=spec-boundary-mismatch

Every turn then re-prefills the entire accumulated conversation. In a real agentic run we watched per-turn prefill climb to 9.5 s at 7,124 tokens while decode sat at 92.9 tok/s — the model was barely getting to generate.

Measured on an 8-turn growing conversation (each turn appends the previous answer, 300 tokens per turn, enable_thinking:false), reporting end-to-end effective throughput — total tokens generated over total wall time, which is what a user actually experiences:

Every arm with MTP enabled took 10 cache misses. Every arm without took 0. Dropping speculation costs ~19 % of single-shot decode and returns +31 % end-to-end, because prefill falls from 36 % of wall time to 9 %.

--spec-mtp-strict-qwen recovers much of the loss but does not eliminate the misses — it is not a way to keep both.

Serving config for agentic / multi-turn

llama-server -m <this-file>.gguf \
  -dev Vulkan0 -ngl 999 -fa on --no-mmap -fit off -c 131072 \
  -ctk q8_0 -ctv q8_0 -np 1 --jinja -cram 4096 \
  --host 0.0.0.0 --port 8080

Size -cram deliberately. It defaults to 8192 MiB per server and scales with context. A 65-token prompt costs ~63.6 MiB of cache state here, so 512 MiB holds about eight short prompts — an agentic conversation evicts itself immediately. 4096 MiB held an 8-turn session with zero misses. On a box running several models, N servers × 8 GiB of default cache will oversubscribe it.

Deployed and re-verified on the seat afterwards: 63.76 tok/s end-to-end, 0 cache misses, prefill 10 % of wall.

Use MTP only for single-shot / batch

The --spec-type draft-mtp numbers elsewhere on this card are single-shot measurements — one request, no conversation history. They are real, and they are the right choice for batch generation, summarisation, or any one-shot call. They do not describe multi-turn behaviour.

---

Head protection — verified, not assumed

Left to their defaults several ftypes emit a 4-bit output.weight, a documented quality-collapse pattern. Every file here was quantised with an explicit head flag and then read back out of the finished file by exact tensor name — never substring-matched, because attn_output matches output and gives a false pass.

On 106 token_embd is deliberately not overridden — Q5_K embeddings are the point of the LEAN tier; only the head needs forcing.

---

Capability control arm

Eight held-out factual questions, enable_thinking default, 1600 tokens, graded on exact content match:

Capability did not move.

---

Reproduction block

binary          llama-server  sha256 d077b98f3f04df4b482a5826c8922e39cda3aa8d2e2b2fbe1870f36d245cebe7
                llama-quantize sha256 1559189968a30781f837e4d25ca3cdf26c8eb0351023675406535eddcbe0db13
source          ROCmFPX @ d3ca53726109b46fd5cd6fa42f688c707079b44e  (-DGGML_HIP=ON -DGGML_VULKAN=ON)
hardware        AMD Ryzen AI Max+ 395 w/ Radeon 8060S (gfx1151), 128 GB unified
gtt_total       131072 MiB
rocm            7.2.4          vulkan  Mesa 25.2.8 (RADV GFX1151), API 1.4.318
kernel          6.17.6-061706-generic      os  Ubuntu 24.04.4 LTS
governor        performance
co-resident     0 seats (box idle for the whole measurement)
env             HSA_OVERRIDE_GFX_VERSION=11.5.1
serve argv      -ngl 999 -fa on -fit off -np 1 -c 4096 --jinja
                --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.0
                -dev {ROCm0|Vulkan0}
quantise argv   llama-quantize <headflag> <BF16> <OUT> <FTYPE> 16
protocol        temperature 0, cache_prompt false, unique nonce, 400 tok,
                1 warm-up discarded, median of 12

SHA256

d50d924221a20973cc19eeaf2b07c9c03bb037076dd6735656781bbdbac66186  Agents-A1-Uncensored-MTP-Q4_0_ROCMFP4_COHERENT.gguf
754deba2a99b05c86fd12ecf220521aac700b27994cb225816f7678911cadd13  Agents-A1-Uncensored-MTP-Q4_0_ROCMFP4_STRIX_LEAN.gguf
ce399fceea98a8766b7d9cf50f76ef31ea3685c7c87de25d216d48c6c50a1f25  Agents-A1-Uncensored-MTP-Q6_0_ROCMFPX_AGENT.gguf
41151e3a6beca3c431e519e638db4a42d537580b54f22789243f0f32686e0ada  Agents-A1-Uncensored-MTP-Q6_0_ROCMFPX_LEAN.gguf