LoRA adapter for zai-org/GLM-5.3-Flash (also applies to zai-org/GLM-5.3-Flash-BF16), fine-tuned on long-form free-form text-adventure / Game Master conversations and character roleplay, with a reasoning (thinking) block on every assistant turn. Compared to cgg507/GLM-5.3-Flash-GM-CoT-LoRA (GM only), v3 adds character roleplay conversations and a darker, harsher antagonist style: NPCs and villains push back, threaten and follow through instead of softening. Same adapter layout as the GM-CoT and E32 adapters (attention, shared experts, dense MLPs and every routed expert). Release checkpoint: step 280 of 288 (97.5 % of the epoch). The final step 288 was a single leftover 1.7k-token document (one noisy full-size Muon update at the lr floor); step 280 was clearly less repetitive in use, so it is the release.
Sibling: cgg507/Ornith-1.5-35B-A3B-REAP20-GM-RP-v3-FP8 (same v3 data on Ornith-1.5-35B-A3B REAP-20).
Experimental. Dark fiction, meant for adult fiction and tabletop play. Uncensored roleplay style: outputs may contain mature themes, graphic violence, cruelty and crude language when the scene calls for it. Not safety-tuned beyond the base model.
| attention / shared expert / dense MLP | rank 64 / alpha 32 (scaling 0.5), 496 adapters |
| routed experts | rank 32 / alpha 16 (scaling 0.5), 36288 adapters (42 layers x 288 experts x 3) |
| KDA linear-attention layers (34) | q_proj k_proj v_proj b_proj forget_gate.f_a_proj forget_gate.f_b_proj g_a_proj g_b_proj o_proj |
| MLA layers (11) | q_a_proj q_b_proj kv_a_proj_with_mqa kv_b_proj o_proj |
| MoE layers (42) | shared expert gate_proj up_proj down_proj + all 288 routed experts gate_proj up_proj down_proj |
| dense layers 0-2 | mlp.gate_proj up_proj down_proj |
| not adapted | routers, DSA indexer, mHC parameters, embeddings, lm_head, vision tower, MTP layer |
| no rsLoRA, no DoRA, dropout 0, bf16 |
| Path | Format |
|---|---|
adapter_model.safetensors, adapter_config.json |
PEFT (transformers module names) |
export_info.json |
how the PEFT file was produced |
nmoe/adapter.safetensors |
nmoe-lora-v1, checkpoint tensor names, attention / shared / dense |
nmoe/adapter_experts.safetensors |
nmoe-lora-v1, per-expert model.layers.N.mlp.experts.E.{gate,up,down}_proj.lora_a/b |
nmoe/adapter_config.json, nmoe/state.json |
trainer config and state |
The PEFT layout (routed experts as target_parameters on the fused gate_up_proj / down_proj, exact
block-diagonal representation) is identical to the E32 / GM-CoT adapters; see the E32 card for the details.
Requires transformers with glm5_next and peft >= 0.21 (target_parameters).
import torch
from transformers import AutoModelForImageTextToText
from peft import PeftModel
base = AutoModelForImageTextToText.from_pretrained(
"zai-org/GLM-5.3-Flash-BF16", dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, "cgg507/GLM-5.3-Flash-GM-RP-v3-LoRA")
# optional: model = model.merge_and_unload()
Use thinking ON (the default GLM-5.3-Flash chat template); the adapter was trained with the reasoning in
<think>...</think> before each reply. The reasoning is long (median ~1.7k characters, like the training data), so
give it a generous reasoning budget (1500+ tokens) or long turns can run out of budget before the reply.
Merging routed-expert deltas directly into existing NVFP4 experts does not work (the deltas are far below the fp4 step); merge into high-precision weights first, then re-quantise.
validate_peft_e32.py --offline (PEFT export checked against the trainer checkpoint, inverse mapping, no base download):
tensors: peft=992 nmoe=992 adapters=496 byte-identical=992/99236288/36288 expert matrices decode bit-exactly (A and B), off-blocks zeroRESULT: PASS--cot-branches): history turns are shown without reasoning (as the
chat template renders them) and each assistant turn is its own target (reasoning + reply + stop), computed with a
shared-history tree (one trunk per conversation, one branch per turn). 5,070 tree documents, 62.9M rows per epoch
instead of 256.4M as separate samples (4.08x less), identical gradients; 29.6M trained tokens.adapter_model.safetensors 18 GBnmoe/adapter_experts.safetensors 14 GBnmoe/adapter.safetensors 575 MBzai-org/GLM-5.3-Flash