GGUF quants of Qwen/Qwen3.5-122B-A10B with the
cgg507/Qwen3.5-122B-A10B-GM-E32-LoRA adapter (checkpoint at step 125 of a 163-step,
2-epoch schedule) merged into the weights: rank-64 adapters on attention (Gated DeltaNet in_proj_qkv / in_proj_z /
out_proj, gated full attention q_proj / k_proj / v_proj / o_proj) and the shared experts, plus rank-32
adapters on all 12,288 routed experts (48 layers x 256). The adapter was fine-tuned on long-form free-form
text-adventure / Game Master roleplay conversations. Sibling model on GLM-5.3-Flash:
cgg507/GLM-5.3-Flash-GM-E32-GGUF.
Experimental. Uncensored roleplay style. Outputs may contain mature themes, graphic violence and crude language when the scene calls for it. Not safety-tuned beyond the base model.
Thinking: keep it disabled. The LoRA was trained without reasoning (assistant turns with an empty / closed
think block). Use a closed think block (<think>\n\n</think>\n\n after <|im_start|>assistant\n), i.e.
enable_thinking: false in the chat template (llama.cpp: --jinja --chat-template-kwargs '{"enable_thinking":false}',
or --reasoning-budget 0).
The MTP (NextN) block is included (unadapted, q8_0). Vision: mmproj/ holds the vision projector converted from the BF16 base (BF16 and Q8_0; the vision tower is not adapted), use it with --mmproj.
| Folder | Size | Parts | Sanity check | Notes |
|---|---|---|---|---|
Q8_0/ |
136.5 GB | 4 | coherent | routed gate/up q8_0, down q8_0 |
Q4_K_M/ |
83.3 GB | 2 | coherent | routed gate/up q4_K, down q5_K |
IQ4_XS/ |
67.0 GB | 2 | coherent | routed gate/up iq3_s, down iq4_xs; imatrix |
IQ2_M/ |
54.2 GB | 2 | coherent | routed gate/up iq2_s, down iq3_s; imatrix |
Multi-part files: download the whole folder and point llama.cpp at the first part (...-00001-of-0000N.gguf).
Attention is never quantized: every Gated DeltaNet projection (attn_qkv, attn_gate, ssm_out, ssm_beta, ssm_alpha) and
every gated full-attention projection stays bf16 (also in the MTP block); the small DeltaNet parameters
(ssm_a, ssm_dt, ssm_conv1d, norms) stay f32 as converted. Only the routed experts are quantized
aggressively, with ffn_down_exps one step above gate/up, mirroring the
AesSedai/Qwen3.5-122B-A10B-GGUF mixes (Q8_0 / Q4_K_M /
IQ4_XS identical expert types; IQ2_M uses the expert types of AesSedai's IQ3_S); AesSedai keeps attention at q8_0,
here it is bf16. The shared expert stays at q8_0 (q6_K in IQ2_M), the MTP block at q8_0.
| Tensor class | Q8_0 | Q4_K_M | IQ4_XS | IQ2_M |
|---|---|---|---|---|
Attention, all layers incl. MTP (Gated DeltaNet attn_qkv, attn_gate, ssm_out, ssm_beta, gated full attention attn_q (q + gate), attn_k, attn_v, attn_output) |
bf16 | bf16 | bf16 | bf16 |
Routed experts gate / up (ffn_gate_exps, ffn_up_exps, blk.0-47) |
q8_0 | q4_K | iq3_s | iq2_s |
Routed experts down (ffn_down_exps, blk.0-47) |
q8_0 | q5_K | iq4_xs | iq3_s |
Shared expert (ffn_{gate,up,down}_shexp, blk.0-47) |
q8_0 | q8_0 | q8_0 | q6_K |
| MTP block blk.48 FFN (routed + shared) | q8_0 | q8_0 | q8_0 | q8_0 |
| token_embd | q8_0 | q8_0 | q8_0 | q6_K |
| output (lm_head) | q8_0 | q8_0 | q8_0 | q6_K |
MTP nextn.eh_proj |
q8_0 | q8_0 | q8_0 | q8_0 |
Norms, routers (ffn_gate_inp, ffn_gate_inp_shexp), ssm_a, ssm_dt, ssm_conv1d |
f32 | f32 | f32 | f32 |
dc4d348443bc).
For every adapter target W += (alpha/r) * B @ A (fp32 compute, bf16 store): 37164 targets found exactly once with matching shape and bf16 dtype (156 attention + 144 shared-expert matrices at scaling 0.5; 36864 routed-expert matrices at scaling 0.5), 39 shards rewritten. The base stores
the routed experts fused (mlp.experts.gate_up_proj [256, 2048, 3072], mlp.experts.down_proj [256, 3072, 1024]);
each expert adapter was applied to its slice (gate = rows 0-1023, up = rows 1024-2047 of gate_up_proj).
Script: scripts/aws_gguf/merge_lora_bf16.py --inplace (refusal_bench).3bb63cbcbfb62056c84fe9e521b0f3d756d29c32 convert_hf_to_gguf.py --outtype bf16 (arch qwen35moe, MTP block
included, experts unfused); vision projector with --mmproj.llama-quantize --tensor-type ... overrides (policy above, generated by
scripts/aws_gguf/quant_policy.py with GGUF_TARGET=qwen; every file's tensor types were verified after
quantization). IQ quants use unsloth's community imatrix for the base model (imatrix_unsloth.gguf from unsloth/Qwen3.5-122B-A10B-GGUF, 76 chunks, 612 entries; copy in
imatrix/). A base-model imatrix is a close approximation for a LoRA-merged model. Files split into parts of at
most 45 GB with llama-gguf-split. Built on an AWS r6id.12xlarge (CPU only).Notes on quality: the LoRA deltas are small. They survive quantization only statistically (as a shift of the rounding decisions), best at Q8_0 / Q4_K_M; at IQ2_M much of the routed-expert fine-tune is below the quantization step. Attention and the shared expert (the rank-64 part) are kept at bf16 / q8_0, so that part survives well. No KLD / perplexity evaluation was run.
Requires llama.cpp with qwen35moe support (built and tested with 3bb63cb).
hf download cgg507/Qwen3.5-122B-A10B-GM-E32-GGUF --include "Q4_K_M/*" --local-dir Qwen3.5-122B-A10B-GM-E32-GGUF
llama-server -m Qwen3.5-122B-A10B-GM-E32-GGUF/Q4_K_M/Qwen3.5-122B-A10B-GM-E32-Q4_K_M-00001-of-0000N.gguf \
-ngl 99 -fa on -c 32768 --jinja --chat-template-kwargs '{"enable_thinking":false}'
The GGUF carries the original Qwen3.5 chat template (ChatML; use --jinja). The adapter was trained with a
Game-Master system prompt; give the model a system prompt describing the scenario and its role. Partial offload:
keep the routed experts on CPU with -ot "exps=CPU" (or --n-cpu-moe N).
See cgg507/Qwen3.5-122B-A10B-GM-E32-LoRA: trained with nmoe-train on the NVFP4 build of the base (r 64 / alpha 32 on attention + shared experts, expert r 32 / alpha 16 on every routed expert, Muon, lr 1e-4 cosine, 2 epochs = 163 steps; this checkpoint: step 125), same Game-Master data as cgg507/GLM-5.3-Flash-GM-E32-LoRA. These quants have not been evaluated beyond the checks above.
IQ2_M/Qwen3.5-122B-A10B-GM-E32-IQ2_M-00001-of-00002.gguf 45 GBIQ2_M/Qwen3.5-122B-A10B-GM-E32-IQ2_M-00002-of-00002.gguf 9.3 GBIQ4_XS/Qwen3.5-122B-A10B-GM-E32-IQ4_XS-00001-of-00002.gguf 45 GBIQ4_XS/Qwen3.5-122B-A10B-GM-E32-IQ4_XS-00002-of-00002.gguf 22 GBQ4_K_M/Qwen3.5-122B-A10B-GM-E32-Q4_K_M-00001-of-00002.gguf 45 GBQ4_K_M/Qwen3.5-122B-A10B-GM-E32-Q4_K_M-00002-of-00002.gguf 39 GBQ8_0/Qwen3.5-122B-A10B-GM-E32-Q8_0-00003-of-00004.gguf 45 GBQ8_0/Qwen3.5-122B-A10B-GM-E32-Q8_0-00002-of-00004.gguf 45 GBQ8_0/Qwen3.5-122B-A10B-GM-E32-Q8_0-00001-of-00004.gguf 45 GBQ8_0/Qwen3.5-122B-A10B-GM-E32-Q8_0-00004-of-00004.gguf 1.7 GBHave the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo cgg507/Qwen3.5-122B-A10B-GM-E32-GGUF ./model-folderQwen/Qwen3.5-122B-A10B