This is a 96 GiB (103 GB) build of orcarouter's abliterated GLM-5.3-Flash, made to fit in the memory of an AMD Strix Halo machine with 128 GB. GLM-5.3-Flash is a 320B mixture-of-experts model from Z.ai that uses 18B of it per token. All 288 experts are kept.
Warning: safety training has been removed from this model. orcarouter's abliteration substantially removes its refusal behaviour, so it can comply with harmful, unethical, offensive or illegal requests. It has no meaningful built-in guardrails. It is meant for research such as refusal and interpretability studies, red-teaming and robustness evaluation. You are fully responsible for how you use it and for what it generates, and you must follow the law that applies to you. Do not put it in front of end users without your own safety and moderation layers. Its outputs do not reflect the views of the uploader, orcarouter or Z.ai.
These files have not been tested on AMD hardware yet, and they were only tested with HarshaVardhan3002/ROCmFPX. Other llama.cpp builds may not recognise the
glm5-nextarchitecture name in these files. Please report speed and quality in the Community tab.Not an official release. All credit for the model goes to Z.ai, and for the abliteration to orcarouter.
Download the IQ2_XXS-E288 folder (three parts) and point the program at the first part.
| File | Size |
|---|---|
GLM-5.3-Flash-Uncensored-StrixHalo-IQ2_XXS-E288-00001-of-00003.gguf |
45.6 GB |
GLM-5.3-Flash-Uncensored-StrixHalo-IQ2_XXS-E288-00002-of-00003.gguf |
46.0 GB |
GLM-5.3-Flash-Uncensored-StrixHalo-IQ2_XXS-E288-00003-of-00003.gguf |
11.6 GB |
Only text generation is included. orcarouter also publishes a vision file (mmproj), but it has not been tested with this build.
This is a heavy compression, and the loss is real. It goes from 318 GiB to 96 GiB.
| Build | Size | Perplexity | KL to Q8 | Same top token as Q8 |
|---|---|---|---|---|
| orcarouter's abliterated Q8_0 (reference) | 318 GiB | 2.61 | ||
| This repo: 2-bit, all 288 experts | 96 GiB | 4.09 | 0.51 | 76.5% |
| Not published: 3-bit, 233 of 288 experts | 99 GiB | 6.01 | 0.91 | 68.8% |
How this was measured: llama-perplexity --kl-divergence against orcarouter's abliterated Q8_0, 40 chunks of 512
tokens of English Wikipedia text (wikitext-2), run on an NVIDIA GB10 GPU. Lower is better for perplexity and KL. "Same top token" is how often this
file's most likely next token matches Q8's.
No full-precision abliterated weights exist, so Q8 is the reference and any loss Q8 itself has is not counted. The text is general English and says nothing direct about coding, reasoning or Chinese.
Short checks on the same GPU, greedy decoding:
IQ2_XXS, down tensors are IQ3_XXS.Build ROCmFPX with Vulkan, then:
llama-server -m IQ2_XXS-E288/GLM-5.3-Flash-Uncensored-StrixHalo-IQ2_XXS-E288-00001-of-00003.gguf -ngl 99 -c 16384 --jinja
Use Vulkan and set the BIOS memory split so the GPU can reach about 110 GiB. HIP/ROCm only allocates from the dedicated part of memory, so a 64/64 split is too small. There is little room left over, so start with a small context and raise it carefully.
MIT. See LICENSE.
IQ2_XXS-E288/GLM-5.3-Flash-Uncensored-StrixHalo-IQ2_XXS-E288-00002-of-00003.gguf 46 GBIQ2_XXS-E288/GLM-5.3-Flash-Uncensored-StrixHalo-IQ2_XXS-E288-00001-of-00003.gguf 46 GBIQ2_XXS-E288/GLM-5.3-Flash-Uncensored-StrixHalo-IQ2_XXS-E288-00003-of-00003.gguf 12 GBorcarouter/GLM-5.3-Flash-Uncensored-GGUF