GLM 5.3 Flash

Q2_K · direct OrcaRouter · Measured configuration

75.64 / 100 Mango weighted score · 54.76 min matched workload · 12/12 mechanical agent passes

The complete official OrcaRouter Q2_K three-shard package of GLM 5.3 Flash Uncensored, with the pinned tokenizer companion linked below.

Benchmark results · Files · Download · Runtime · Provenance · Full collection

Join Infinity on Discord

Get model support, request models and quants, discuss benchmarks, and follow new releases and model improvements.

Join the Infinity community

All model tests, per-question scores, experiment history and scoring methods

Qwen effort study — separate full-404 cohort: medium 82.87/100 versus xhigh 84.41/100; paired difference +1.54 points (95% interval [-1.71, +4.82]). 404 collected questions, 403 valid questions graded per effort; one invalid reference annotation excluded symmetrically. The historical 202-question cross-model comparison below is unchanged. Study, timing and blind subjective results.

Preserved and documented by isilicon / Mango. The original model, modification and quantization remain the upstream creators' work. This repository adds attributed preservation and exact-configuration evaluation, with the selected source bytes unchanged.

At a glance

GB is decimal and GiB is binary. Payload size is disk/storage size, not a claim that the model fits in the same amount of VRAM. Runtime, KV cache, working memory and companions add their own requirements.

Mango evaluation

The exact official Q2_K package scored 75.64/100 in 54.76 recorded workload minutes, with no infrastructure errors or length finishes. All 12 agent-loop cases passed the mechanical checks; its function-call score was 23/40, so the combined agentic score is 78.75 rather than 100.

Local coding tests passed 16/20 and strict IFEval passed 13/20. Pure EQ scored 81.64, while the combined social/EQ score was 68.60 because theory-of-mind labels passed 10/18. These checks do not establish broad semantic judgment or live agent competence.

Compared with Bolding IQ2_XXS, this run scored 3.47 points lower and took 1.34× as long (54.76 versus 40.98 minutes). The paired 95% difference interval, [-7.86, +1.01], crosses zero. It does not establish a quality winner between those two packages, and quantization plus engine changes prevent attributing the gap to publisher identity.

The previous mradermacher Flash Next configuration scored 8.55 points higher and used 35.33 workload minutes. Q2_K minus Flash has a paired 95% interval of [-13.70, -3.54]. This sample supplies no quality or speed case for promoting Q2_K over Flash; the existing economics choice remains unchanged.

Domain scores equally average their component task means. The overall score applies the weights above. EQ is continuous; the 12-case mechanical agent pass rate is only one component of the agentic domain.

Task-level quality and uncertainty

These are small released subsets and local graders. HumanEval/BFCL scores are not the complete official benchmark pipelines. Agent checks cover completion, evidence, permissions and applicable pricing totals; other answer semantics remain unjudged.

Recorded performance

Workload time excludes loading, restarts, wipe downtime and idle gaps. Latency/TTFT summaries cover suite requests only. The reported token rate divides engine usage counts by request time; it is not native decode throughput and token accounting differs across engines. No concurrent-serving result is implied.

Matched cohort comparison

The old mixed GLM row is a historical, internally capped baseline whose exact weights are unavailable. Other rows also differ in quant, engine, context and effort. Point ranks do not settle the quality ordering.

Paired score differences

Intervals use 10,000 paired task-stratified bootstrap resamples, seed 41, with fixed task/domain weights. An interval containing zero does not establish a quality winner. See methodology for scope and definitions.

Download the current full numerical report and methodology. The numerical report contains aggregate statistics and hashes; no prompts or generated answers are published.

Runtime and compatibility

Its GGUF declares glm5next. Stock restored builds expecting glm5_next rejected it before inference. Mango recovered compatibility with unslothai/llama.cpp branch glm5next/upstream at revision 86ebfef2c6a0f3359a2a07d2c215d61b0fa885c9. Do not rename or edit verified GGUF architecture headers.

32,768 tokens; requested/applied native high effort on all 202 suite requests; thinking enabled with reasoning budget -1, meaning no artificial thinking limit.

The local evaluation covers text/tool tasks. It does not establish the full advertised context ceiling, image/audio quality or every optional speculative-decoding path. Publisher capabilities and Mango-verified behavior are separate scopes.

Companion artifacts

The selected payloads of this model and its linked companions total 116.89 GB before runtime/cache files. That is a download/storage total, not a RAM/VRAM fit guarantee.

Files and integrity

Full paths, byte sizes and complete SHA-256/Git blob identities are in SOURCE_MANIFEST.json. Available original README/license/notice files are also preserved. MIRROR_STATUS.json records the earlier source-file verification; MODEL_CARD_STATUS.json records this documentation refresh separately.

Download and use

The commands pin the already-verified mirror weight revision. Documentation can advance on main without changing that preserved weight snapshot. Keep original subfolders and all shards.

python -m pip install -U huggingface_hub

hf download isilicon/mango-glm53flash-orca-q2k-20261029 \
  Q2_K/GLM-5.3-Flash-Uncensored-Q2_K-00001-of-00003.gguf \
  Q2_K/GLM-5.3-Flash-Uncensored-Q2_K-00002-of-00003.gguf \
  Q2_K/GLM-5.3-Flash-Uncensored-Q2_K-00003-of-00003.gguf \
  --revision bf8d44cea44c7d90dc11b4fa3fe89a776b2a4310 \
  --local-dir ./weights/glm53flash-orca-q2k-20261029

hf download isilicon/mango-glm53flash-orca-tokenizer-20261029 \
  tokenizer.json \
  tokenizer_config.json \
  --revision 5a600e5b7353ac17522267ca3e76057bfe21a394 \
  --local-dir ./weights/glm53flash-orca-tokenizer-20261029

Load example and setup scope

llama-server \
  --model ./weights/glm53flash-orca-q2k-20261029/Q2_K/GLM-5.3-Flash-Uncensored-Q2_K-00001-of-00003.gguf \
  --host 127.0.0.1 --port 8080 --ctx-size 32768 \
  --jinja --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0

This is a minimal text-serving example for a compatible build, not a replay of every benchmark fitting/offload flag. The recorded configuration and engine caveats define the tested scope.

Provenance and license

This is a different quant/package from Bolding IQ2_XXS and the unavailable historical mixed GLM. Their benchmark scores cannot be assigned to this Q2_K release. The upstream card covers other quants and modalities beyond the files preserved here.

Original notices, base-model conditions and the publisher's stated limitations still apply. Downloadable preservation does not add new license rights. Source-described refusal modification and publisher-side benchmark claims have not been independently re-measured by this Mango task cohort.

Original publisher documentation

The unchanged upstream README and pinned source card retain the original usage guidance, technical discussion, attribution, caveats and citations.

Read the complete original publisher card

Source reference. The following is the original publisher's documentation. Its other quants, full-model files, context claims and benchmark tables can exceed the selection in this mirror. Relative asset/file URLs are qualified to the pinned original source; publisher measurements remain separate from Mango measurements.

GLM-5.3-Flash-Uncensored-GGUF

GGUF quants (2-bit → 6-bit) of the abliterated (refusal-removed) GLM-5.3-Flash — for llama.cpp

One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.

Website · Model Catalog · GitHub · Discord · X

---

GGUF conversions of the abliterated (refusal-removed) build of Z.ai's GLM-5.3-Flash — a 320B / 18B-active Mixture-of-Experts model (glm5_next, 288 routed experts top-8 + 1 shared) with hybrid linear + sparse attention, 4-wide Manifold-Constrained Hyper-Connections, a native vision + video tower, and a 1M-token context. These files run in llama.cpp (CPU / CUDA / Metal / ROCm), quantized from 2-bit to 6-bit, with a separate mmproj file that restores vision. Browse all models in the OrcaRouter Model Catalog.

Derived releases:  •  GLM-5.3-Flash-Uncensored-FP8 (block-FP8, mirrors official)  •  GLM-5.3-Flash-Uncensored-NVFP4 (experts-only NVFP4, for vLLM)  •  GLM-5.3-Flash-Uncensored-MLX (2/3/4/6-bit, Apple Silicon)  •  GLM-5.3-Flash-Uncensored-GGUF (this repo).

---

⚠️ Disclaimer — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). As a direct consequence:

• It will comply with harmful, unethical, offensive, or illegal requests that the original GLM-5.3-Flash would refuse. It has no meaningful built-in guardrails.
• It is released strictly for legitimate research — interpretability, AI-safety and refusal-mechanism study, red-teaming, robustness evaluation, and controlled experiments.
• You assume full responsibility and liability for how you use it and for everything it generates. Do not deploy it to end users or in production without adding your own safety, moderation, and abuse-prevention layers.
• Use must comply with the MIT License inherited from the base model, and all laws and regulations that apply to you.
• The authors and uploaders accept no liability for any misuse or harm. Its outputs do not reflect the views of the uploaders or of Z.ai / Zhipu AI.

By downloading or using this model you acknowledge and accept the above.

---

Requirements — build a recent llama.cpp

The glm5_next (GLM-5.3-Flash) architecture is new and not yet in mainline llama.cpp — support is in PR #27754 (model: add GLM-5-Next (GLM-5.3-Flash)). Until it merges, build from the PR branch; a bundled/older runtime will fail with unknown architecture 'glm5next'.

git clone -b glm5next/upstream https://github.com/unslothai/llama.cpp
cmake llama.cpp -B llama.cpp/build -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first \
  --target llama-cli llama-server llama-mtmd-cli llama-gguf-split

(Drop -DGGML_CUDA=ON for a CPU-only build.) transformers>=5.16 is needed only if you convert from the HF weights yourself.

Note — no MTP head. These GGUFs do not include the multi-token-prediction (mtp) speculative draft head (layer 45); the llama.cpp glm5next backend has no MTP inference path, so the converter drops it. All text + vision capability is unaffected. For speculative decoding, use the MLX build orcarouter/GLM-5.3-Flash-Uncensored-MLX.

Files

Files larger than ~48 GB are split with llama-gguf-split into parts — download the whole set for a quant and point llama.cpp at the …-00001-of-000NN.gguf part; it loads the rest automatically. Sizes are approximate.

For full precision, use the block-FP8 safetensors in orcarouter/GLM-5.3-Flash-Uncensored-FP8.

Usage (llama.cpp — PR #27754 build)

Download

hf download orcarouter/GLM-5.3-Flash-Uncensored-GGUF \
  --include "Q4_K_M/*" "mmproj-*" --local-dir ./glm53-flash-uncensored

Chat (text)

./llama-cli -m Q4_K_M/GLM-5.3-Flash-Uncensored-Q4_K_M-00001-of-00005.gguf --jinja -c 8192 -p "Hello!"

OpenAI-compatible server (vision + reasoning)

./llama-server -m Q4_K_M/GLM-5.3-Flash-Uncensored-Q4_K_M-00001-of-00005.gguf \
  --mmproj mmproj-GLM-5.3-Flash-Uncensored-F16.gguf \
  --host 0.0.0.0 --port 8000 -c 8192 --jinja

• Vision: pass --mmproj …, then send OpenAI image_url content parts (base64 data-URI or URL).
• Reasoning: GLM-5.3-Flash has no enable_thinking toggle — its chat template always opens a <think> block. Give generation enough budget to reach </think> or the final answer is truncated inside the scratchpad.

Evaluation — quality vs the source

Perplexity, KL-divergence and top-1 token agreement of each quant against the near-lossless Q8_0 reference (the source dequantized to 8-bit and run through the identical llama.cpp forward, so the quantization is the only variable). Measured on wikitext-2 (~91k tokens, n_ctx=512). Lower KLD and higher Top-1 = closer to the source.

All three signals agree on the ordering and on where the cliff is: Q6_K is within noise of the source, Q4_K_M is the sweet spot, Q3_K_M degrades gently, and Q2_K costs a lot — pick it for fit, not quality.

Uncensoring

Abliteration is baked into the weights and inherited by every quant. On the held-out JailbreakBench test split, harmful-prompt refusal drops from ~89 % on the base GLM-5.3-Flash to the low-teens, with benign over-refusal at 0 % (see the base GLM-5.3-Flash-Uncensored-FP8 card for the full safety numbers). Refusal is reduced, not removed — a few extreme content categories the single direction does not reach still refuse.

Hardware

Runs on CPU, CUDA, Metal, or ROCm via llama.cpp. As a large MoE, only 8 of 288 routed experts are active per token, so decode is far faster than the total parameter count suggests, but the full weights must fit in RAM/VRAM (or be memory-mapped): budget ≈ the file size + KV cache + (for vision) the ~1 GB mmproj. Multi-GPU (-ngl split) and CPU+GPU offload both work.

License

MIT, inherited from zai-org/GLM-5.3-Flash. Abliteration and quantization do not change the underlying license obligations.

Preservation record

Selected source bytes are verified against pinned metadata; the original eight graded mirrors also passed the previously recorded anonymous 1 MiB weight-prefix checks. This is not a full independent-device restore/load test. Hugging Face repository copies share a provider and do not guarantee permanent storage.

Collection · Source manifest · Original documentation