GGUF quants of Qwen/Qwen3.5-122B-A10B with the
cgg507/Qwen3.5-122B-A10B-GM-CoT-LoRA adapter (checkpoint at step 145 of a
145-step, 1-epoch schedule) merged into the weights: rank-64 adapters on attention (Gated DeltaNet
in_proj_qkv / in_proj_z / out_proj, gated full attention q_proj / k_proj / v_proj / o_proj) and the
shared experts, plus rank-32 adapters on all 12,288 routed experts (48 layers x 256). This is the CoT / reasoning
variant of the Game-Master roleplay adapter: every assistant turn was trained with a chain of thought, so the model
reasons in <think> about the scene, the system prompt and the player's action before writing the reply. The
no-thinking sibling (same base, same data, empty think blocks): cgg507/Qwen3.5-122B-A10B-GM-E32-GGUF.
Experimental. Uncensored roleplay style. Outputs may contain mature themes, graphic violence and crude language when the scene calls for it. Not safety-tuned beyond the base model.
Thinking: keep it ON. The LoRA was trained with reasoning on every turn (generation prompt ends in
<|im_start|>assistant\n<think>\n, the model writes its reasoning, then </think>\n\n and the reply). That is the
default of the Qwen3.5 chat template - just use --jinja and do not pass enable_thinking: false.
The adapter was trained against an abliterated NVFP4 build of the base, but is merged here into the standard (non-abliterated) BF16 base on purpose: in our testing the abliterated model falls into heavy repetition when pushed into content that would normally trigger a refusal, and the adapter works better on the standard weights. See the LoRA card for details.
The MTP (NextN) block is included (unadapted, q8_0). Vision: mmproj/ holds the vision projector converted from the BF16 base (BF16 and Q8_0; the vision tower is not adapted), use it with --mmproj.
| Folder | Size | Parts | Sanity check | Notes |
|---|---|---|---|---|
Q8_0/ |
136.5 GB | 4 | coherent | routed gate/up q8_0, down q8_0 |
Q4_K_M/ |
83.3 GB | 2 | coherent | routed gate/up q4_K, down q5_K |
IQ4_XS/ |
67.0 GB | 2 | coherent | routed gate/up iq3_s, down iq4_xs; imatrix |
IQ2_M/ |
54.2 GB | 2 | coherent | routed gate/up iq2_s, down iq3_s; imatrix |
Multi-part files: download the whole folder and point llama.cpp at the first part (...-00001-of-0000N.gguf).
Attention is never quantized: every Gated DeltaNet projection (attn_qkv, attn_gate, ssm_out, ssm_beta, ssm_alpha) and
every gated full-attention projection stays bf16 (also in the MTP block); the small DeltaNet parameters
(ssm_a, ssm_dt, ssm_conv1d, norms) stay f32 as converted. Only the routed experts are quantized
aggressively, with ffn_down_exps one step above gate/up, mirroring the
AesSedai/Qwen3.5-122B-A10B-GGUF mixes (Q8_0 / Q4_K_M /
IQ4_XS identical expert types; IQ2_M uses the expert types of AesSedai's IQ3_S); AesSedai keeps attention at q8_0,
here it is bf16. The shared expert stays at q8_0 (q6_K in IQ2_M), the MTP block at q8_0.
| Tensor class | Q8_0 | Q4_K_M | IQ4_XS | IQ2_M |
|---|---|---|---|---|
Attention, all layers incl. MTP (Gated DeltaNet attn_qkv, attn_gate, ssm_out, ssm_beta, gated full attention attn_q (q + gate), attn_k, attn_v, attn_output) |
bf16 | bf16 | bf16 | bf16 |
Routed experts gate / up (ffn_gate_exps, ffn_up_exps, blk.0-47) |
q8_0 | q4_K | iq3_s | iq2_s |
Routed experts down (ffn_down_exps, blk.0-47) |
q8_0 | q5_K | iq4_xs | iq3_s |
Shared expert (ffn_{gate,up,down}_shexp, blk.0-47) |
q8_0 | q8_0 | q8_0 | q6_K |
| MTP block blk.48 FFN (routed + shared) | q8_0 | q8_0 | q8_0 | q8_0 |
| token_embd | q8_0 | q8_0 | q8_0 | q6_K |
| output (lm_head) | q8_0 | q8_0 | q8_0 | q6_K |
MTP nextn.eh_proj |
q8_0 | q8_0 | q8_0 | q8_0 |
Norms, routers (ffn_gate_inp, ffn_gate_inp_shexp), ssm_a, ssm_dt, ssm_conv1d |
f32 | f32 | f32 | f32 |
dc4d348443bc).
For every adapter target W += (alpha/r) * B @ A (fp32 compute, bf16 store): 37164 targets found exactly once with matching shape and bf16 dtype (156 attention + 144 shared-expert matrices at scaling 0.5; 36864 routed-expert matrices at scaling 0.5), 39 shards rewritten. The base stores
the routed experts fused (mlp.experts.gate_up_proj [256, 2048, 3072], mlp.experts.down_proj [256, 3072, 1024]);
each expert adapter was applied to its slice (gate = rows 0-1023, up = rows 1024-2047 of gate_up_proj).
Script: scripts/aws_gguf/merge_lora_bf16.py --inplace (refusal_bench).3bb63cbcbfb62056c84fe9e521b0f3d756d29c32 convert_hf_to_gguf.py --outtype bf16 (arch qwen35moe, MTP block
included, experts unfused); vision projector with --mmproj.llama-quantize --tensor-type ... overrides (policy above, generated by
scripts/aws_gguf/quant_policy.py with GGUF_TARGET=qwen_cot; every file's tensor types were verified after
quantization). IQ quants use unsloth's community imatrix for the base model (imatrix_unsloth.gguf from unsloth/Qwen3.5-122B-A10B-GGUF, 76 chunks, 612 entries; copy in
imatrix/). A base-model imatrix is a close approximation for a LoRA-merged model. Files split into parts of at
most 45 GB with llama-gguf-split. Built on an AWS r6id.12xlarge (CPU only).Notes on quality: the LoRA deltas are small. They survive quantization only statistically (as a shift of the rounding decisions), best at Q8_0 / Q4_K_M; at IQ2_M much of the routed-expert fine-tune is below the quantization step. Attention and the shared expert (the rank-64 part) are kept at bf16 / q8_0, so that part survives well. No KLD / perplexity evaluation was run.
Requires llama.cpp with qwen35moe support (built and tested with 3bb63cb).
hf download cgg507/Qwen3.5-122B-A10B-GM-CoT-GGUF --include "Q4_K_M/*" --local-dir Qwen3.5-122B-A10B-GM-CoT-GGUF
llama-server -m Qwen3.5-122B-A10B-GM-CoT-GGUF/Q4_K_M/Qwen3.5-122B-A10B-GM-CoT-Q4_K_M-00001-of-0000N.gguf \
-ngl 99 -fa on -c 32768 --jinja
The GGUF carries the original Qwen3.5 chat template (ChatML; use --jinja). Thinking is on by default
(enable_thinking: true) - that is how this adapter is meant to run; the model writes its Game-Master reasoning
before the reply. The adapter was trained with a Game-Master system prompt; give the model a system prompt
describing the scenario and its role. Partial offload: keep the routed experts on CPU with -ot "exps=CPU"
(or --n-cpu-moe N). Give it enough context / max tokens for the reasoning plus the reply.
See cgg507/Qwen3.5-122B-A10B-GM-CoT-LoRA: trained with nmoe-train (--cot-branches --expert-rank 32)
against an abliterated NVFP4 build of the base (r 64 / alpha 32 on attention + shared experts, expert r 32 /
alpha 16 on every routed expert, Muon, lr 2e-4 cosine). 731 GM roleplay conversations, 17,449 assistant turns,
a chain of thought on every turn (reasoning written by GLM-5.3-Flash-GM, aligned to the actual reply), 24.5M
trained tokens, max sequence 49,152, 1 epoch = 145 steps; loss 1.42 -> 1.165 (perplexity ~4.1 ->
~3.25 on trained tokens); this checkpoint: step 145 (final). Same Game-Master data as the no-thinking
cgg507/Qwen3.5-122B-A10B-GM-E32-LoRA adapter. These quants have not been evaluated beyond the
checks above.
IQ2_M/Qwen3.5-122B-A10B-GM-CoT-IQ2_M-00001-of-00002.gguf 45 GBIQ2_M/Qwen3.5-122B-A10B-GM-CoT-IQ2_M-00002-of-00002.gguf 9.3 GBIQ4_XS/Qwen3.5-122B-A10B-GM-CoT-IQ4_XS-00001-of-00002.gguf 45 GBIQ4_XS/Qwen3.5-122B-A10B-GM-CoT-IQ4_XS-00002-of-00002.gguf 22 GBQ4_K_M/Qwen3.5-122B-A10B-GM-CoT-Q4_K_M-00001-of-00002.gguf 45 GBQ4_K_M/Qwen3.5-122B-A10B-GM-CoT-Q4_K_M-00002-of-00002.gguf 39 GBQ8_0/Qwen3.5-122B-A10B-GM-CoT-Q8_0-00003-of-00004.gguf 45 GBQ8_0/Qwen3.5-122B-A10B-GM-CoT-Q8_0-00002-of-00004.gguf 45 GBQ8_0/Qwen3.5-122B-A10B-GM-CoT-Q8_0-00001-of-00004.gguf 45 GBQ8_0/Qwen3.5-122B-A10B-GM-CoT-Q8_0-00004-of-00004.gguf 1.7 GBHave the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo cgg507/Qwen3.5-122B-A10B-GM-CoT-GGUF ./model-folder