GLM-5.3-Flash-Abliterated-MLX-4bit
A high-quality abliterated 4-bit build of GLM-5.3-Flash, optimized for Apple Silicon with oMLX.
Lineage: zai-org/GLM-5.3-Flash → Blackfrost-Research/GLM-5.3-Flash-DERISKED-BF16 (abliterated) → this build (4-bit oQ4e).
The vision tower and the MTP (multi-token-prediction) draft head are both preserved and working, so the build retains image input and speculative decoding. Quantized with oMLX to oQ4e: iMatrix-calibrated mixed precision, with per-tensor bit widths rather than a uniform 4 bits. This model works standalone on stock oMLX with no extra downloads; the speculative-decoding speedups (MTP and DFlash2) are optional set-ups described below.
| This build: 4-bit | |
|---|---|
| DFlash2 decode | Short, Thinking-Max: 29.5 - 34.8 tok/s Short, Thinking-High: 39.3 - 51.2 tok/s Long Agentic, Thinking-Max: 33.4 - 33.6 tok/s Long Agentic, Thinking-High: 31.9 - 32.8 tok/s |
| MTP decode | Short, Thinking-Max: 30.4 - 30.9 tok/s Short, Thinking-High: 35.3 - 39.7 tok/s Long Agentic, Thinking-Max: 27.6 - 29.4 tok/s Long Agentic, Thinking-High: 26.7 - 29.6 tok/s |
| AR (no speculative decode) | Short: 29.3 - 29.7 tok/s, flat Long Agentic: 26.5 - 26.6 tok/s, flat |
| Abliteration | 1.3% refusals, measured by Blackfrost-Research on upstream release family |
High is the recommended thinking level for most use cases. As described in the X post from zainhas, testing on Z.ai Code Bench v1.0 shows that GLM-5.3-Flash, High and Max sit at basically the same accuracy, while Max uses about 2× the output tokens (70k vs 140k).
DFlash2 is the fastest option at every context length once its prefix cache works. On the stock GLM adapter the cache is disabled, so DFlash2 re-prefills the whole prompt on every long-context request and MTP or AR win there instead. Our prefix-cache patch (publishing to GitHub shortly; PR pending upstream) fixes that: multi-turn long-context TTFT drops ~46 s to 0.26 s, wall-clock 61.8 s to 15.1 s, and patched DFlash2 then beats both MTP and AR. The warm hit is measured at both temperature 0 and temperature 1. It is not universal - one longer-generation run re-prefilled instead of hitting, and the trigger is still under investigation - so treat the warm numbers as best-case until that is pinned down. Decode rates are unchanged by the patch. oMLX will likely add this natively soon - the fork is only needed if you want DFlash2 with warm caching today.
Wallclock to generate 1024 tokens, short context (temp 0 - temp 1 ranges):
| decode type | Thinking-High | Thinking-Max | High saves |
|---|---|---|---|
| DFlash2 + prefix-cache patch (temp 0) | 21.0 s | 35.4 s | -41% |
| DFlash2 (stock adapter) | 20.8 - 27.5 s | 30.4 - 35.7 s | -10% to -42% |
| MTP | 26.8 - 30.2 s | 34.2 - 34.8 s | -13% to -22% |
| AR | 35.8 - 36.0 s | 35.5 - 35.8 s | ~0% |
Short prompts have no long prefill to cache, so the patch neither helps nor hurts here - DFlash2 already wins.
Wallclock to generate 512 tokens, Long Agentic context (temp 0 - temp 1 ranges):
| decode type | prompt cache state | Thinking-High | Thinking-Max | High saves |
|---|---|---|---|---|
| DFlash2 + prefix-cache patch (temp 0 measured) | warm | 15.1 s | 15.9 s | -5% |
| DFlash2 (stock adapter) | none (full prefill every request) | 61.8 - 62.1 s | 61.3 - 61.4 s | ~+1% (Max faster) |
| MTP | warm | 22.2 - 24.0 s | 22.3 - 23.4 s | mixed, ±5-8% |
| AR | warm | 24.1 - 24.8 s | 24.1 - 25.1 s | ~0% |
The patched row is measured at temperature 0; the warm hit itself is confirmed at both temperatures (0.26 s TTFT each). One longer-generation run missed and re-prefilled, so the warm figures are best-case pending that investigation.
At equal output length; in real use High's advantage roughly doubles again, since Max generates about 2× the tokens per answer.
| download size | 173 GiB across 35 shards |
| resident when serving | ~190 GiB |
| tensors | 3057 (59 MTP · 347 vision) |
| base model | 320B parameters, ~18B active per token (MoE, 8 of 288 experts + 1 shared) |
| architecture | glm5_next: 45 layers, hybrid KDA (recurrent) + DSA (sparse attention), native multimodal |
| context | 1,048,576 tokens (set your serving window to what your RAM allows) |
⚠️ Intended use and terms
Published for experimental AI research and AI-safety research only: red-teaming, refusal and alignment research, interpretability, and quantization or inference research.
It must not be used for anything illegal, in any jurisdiction, under any circumstances. The removal of refusal behavior is a research property of the artifact, not an invitation, a permission, or an endorsement.
You are solely responsible for your use of it and for complying with applicable laws and upstream licence terms. It ships with essentially no safety refusals: assume it will comply with any instruction, including harmful ones. Run it locally on hardware you control. Do not expose it as a public endpoint or deploy it to untrusted users.
Performance
Measured on an M3 Ultra Mac Studio (256 GB) with oMLX 0.6.3. Two workloads: short (a code-continuation prompt, 1024 tokens generated) and agentic (a real 14-message tool-calling transcript, ~20.4k GLM tokens of context, 512 generated) - the same benchmark used for the sibling Qwen builds. TTFT counts the first token of reasoning, not just the answer. Temp 1.0 rows use the model's recommended sampling (top_p 0.95).
GLM-5.3-Flash reasons at three levels - low, high, and max (the
default) - set via reasoning_effort. There is no native thinking-off: the chat
template always opens a <think> block. The "off" rows below force the block
closed (<think></think> prefix via the completions API), which is the only
true no-thinking mode this model has.
Decode throughput below (DFlash2 rows use the default adaptive verify; see the tuning note for strict-verify, which is faster on short context). TTFT is 0.85–1.0 s on short prompts. At 20.4k context, cold prefill runs ~445 tok/s (~46 s); AR and MTP then ride the shared prompt cache (~4.8 s warm), while DFlash2 on the stock adapter repays the full prefill on every request (see limitations).
SHORT context (code prompt, 1024 tokens generated)
AR baseline (no speculation): 29.3–29.7 tok/s, flat across all thinking levels and temperatures.
| Thinking level | decode type | decode tok/s (temp 0) | gain over AR (temp 0) | decode tok/s (temp 1) | gain over AR (temp 1) |
|---|---|---|---|---|---|
| Max | DFlash2 | 29.5 | −0.7% | 34.8 | +18.4% |
| Max | MTP | 30.9 | +4.0% | 30.4 | +3.4% |
| High | DFlash2 | 51.2 | +74.1% | 39.3 | +34.1% |
| High | MTP | 39.7 | +35.0% | 35.3 | +20.5% |
| Low | DFlash2 | 49.3 | +66.6% | 48.2 | +63.4% |
| Low | MTP | 40.2 | +35.8% | 36.5 | +23.7% |
| Off* | DFlash2 | 54.0 | +83.7% | 50.2 | +71.3% |
| Off* | MTP | 39.8 | +35.4% | 37.3 | +27.3% |
Long Agentic context (real 14-message tool transcript, ~20.4k tokens, 512 generated)
AR baseline (no speculation): 26.5–26.6 tok/s, flat across all thinking levels and temperatures.
| Thinking level | decode type | decode tok/s (temp 0) | gain over AR (temp 0) | decode tok/s (temp 1) | gain over AR (temp 1) |
|---|---|---|---|---|---|
| Max | DFlash2 | 33.6 | +26.8% | 33.4 | +26.0% |
| Max | MTP | 29.4 | +10.9% | 27.6 | +4.2% |
| High | DFlash2 | 31.9 | +20.4% | 32.8 | +23.8% |
| High | MTP | 26.7 | +0.8% | 29.6 | +11.7% |
| Low | DFlash2 | 33.0 | +24.1% | 29.3 | +10.6% |
| Low | MTP | 28.7 | +7.9% | 27.5 | +3.8% |
| Off* | DFlash2 | 31.5 | +18.4% | 32.7 | +22.9% |
| Off* | MTP | 29.4 | +10.5% | 26.7 | +0.4% |
* No native thinking-off exists in the GLM-5.3 template - Off rows use the
forced-close method (<think></think> prefix via the completions API).
Recommended configurations (measured):
| use case | config | decode |
|---|---|---|
| Short / interactive | DFlash2, verify_mode: "dflash" (strict) |
56.4 off · 51.8 low · 50.6 high · 33.2 max |
| One-shot long context | DFlash2, verify_mode: "adaptive" |
33.6 at 20.4k |
| Repeated long-context chat | DFlash2 + prefix-cache patch (0.32 s warm TTFT), or MTP on stock patches (4.8 s warm TTFT) |
Strict verify wins short prompts (full speculation blocks pay off on high-acceptance content, +4–13 % over adaptive); adaptive wins past ~8k context. Both verify modes produce byte-identical output to each other.
Quality: plain decoding is bit-deterministic (verified across restarts). Speculative output is deterministic and greedy-equivalent up to floating-point tie-breaks - roughly one benign fork per few hundred characters where a near-tie resolves differently and both continuations are valid; it is not bit-identical to non-speculative output. Verified by reading divergences, not just hashing.
Speculative decoding: two optional speed-ups
The model works standalone. For faster decode you can enable MTP (the draft head inside this repo - no extra download) or DFlash2 (a separate 2.3 GB drafter - bigger speed-up). Both are greedy-equivalent up to floating-point tie-breaks (see Quality above) - a different but equally valid answer, not a worse one.
Both currently require two oMLX patch sets on 0.6.3, from upstream PR branches (#3228 - GLM MTP runtime - and #3261 - GLM DFlash2 adapter; #3261 is maintained as a closed branch). Follow each PR's install notes, then:
MTP - in ~/.omlx/model_settings.json under this model's entry:
{"mtp_enabled": true}
DFlash2 - download the drafter and point the settings at it:
hf download incoai/GLM-5.3-Flash-DFlash2 --local-dir ~/omlx-models/GLM-5.3-Flash-DFlash2
{
"dflash_enabled": true,
"dflash_draft_model": "~/omlx-models/GLM-5.3-Flash-DFlash2",
"dflash_block_size": 8,
"dflash_verify_mode": "dflash",
"dflash_draft_quant_enabled": true,
"dflash_draft_quant_weight_bits": 8,
"dflash_draft_quant_group_size": 64
}
Use "dflash_verify_mode": "adaptive" instead if your prompts routinely
exceed ~8k tokens. The weight_bits: 8 drafter quantization is measured
fastest on this build - it cuts draft cost with zero acceptance loss (2-bit
collapses; leave it at 8). Restart the server after changing settings. Enable
one speculation mode at a time.
A note on expectations: headline speculative-decoding numbers you may have seen elsewhere (e.g. ~80 tok/s) come from synthetic benchmarks whose prompt is one paragraph repeated hundreds of times - the easiest possible input for a drafter. The numbers on this card are from real prompts. A stock-weights conversion measured with the same recipe and settings splits the lead with this build by thinking mode: stock decodes notably faster at default max-effort thinking (up to +29% short-context with DFlash2), while this build is faster at thinking off/high/low (+2 to +10%) - which includes all the fastest configurations on this card. Raw non-speculative speed is identical between the two.
Abliteration and refusal surface
Refusal behaviour was removed at the weight level in the upstream BF16 by Blackfrost-Research (their "DWM" method; the MTP layer was transplanted back from the clean checkpoint). No persona or system prompt ships with this build - the template is the stock GLM-5.3 template plus multimodal/tool-call fixes, and it injects nothing.
Blackfrost-Research's reported figures for the release family, measured on their behavior-matched NVFP4 deployment checkpoint with R1-HARMFUL-BENCH-450 under a bare chat configuration (thinking enabled, maximum reasoning effort, temperature 1.0, top-p 0.95, top-k omitted, max 16,384 output tokens; responses reviewed after generation to separate actual refusals from false-positive string matches):
| Evaluation slice | Final judged refusals |
|---|---|
| Harmful prompts | 4 / 300 (1.3%) |
| Full suite | 5 / 450 (1.1%) |
| API errors | 0 / 450 |
These values are upstream measurements of a different deployment checkpoint, not of this build. Refusals were spot-checked after quantization (the derisked behaviour survives), but no systematic refusal benchmark has been run on this 4-bit build. "Abliterated" describes the upstream weights, not a property this conversion re-measured.
Quick start
Requires oMLX on Apple Silicon (256 GB recommended).
hf download grant-ai/GLM-5.3-Flash-Abliterated-MLX-4bit \
--local-dir ~/omlx-models/GLM-5.3-Flash-Abliterated-MLX-4bit
omlx serve
curl http://127.0.0.1:8153/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model": "GLM-5.3-Flash-Abliterated-MLX-4bit",
"messages": [{"role": "user", "content": "Explain MoE routing."}],
"max_tokens": 1024, "temperature": 1.0, "top_p": 0.95}'
Settings that matter
| setting | value |
|---|---|
| sampler | temperature 1.0, top_p 0.95 (from the model's generation_config.json) |
| reasoning effort | chat_template_kwargs: {"reasoning_effort": "low" | "high"} - anything else means max (the default) |
| thinking off | not natively supported - the template always opens <think>. True off requires the completions API with a <think></think> prefix (see Performance) |
| multi-turn chat | pass chat_template_kwargs: {"clear_thinking": true} per the upstream model card |
| MTP / DFlash2 | optional; see Speculative decoding above |
reasoning_effort is the biggest lever on time-to-answer: it controls how many
reasoning tokens are generated before the answer, while plain decode speed
stays flat. With speculation enabled, per-token speed depends on how
predictable the generated text is, not on the effort setting itself - on
code, off/low/high all decode far faster than max (whose deliberative
reasoning drafts poorly), while on methodical math reasoning max can be the
fastest. Note that low is a soft request the model sometimes ignores, not a
hard budget.
Technical details
| architecture | glm5_next - 45 layers: 34 KDA (gated-delta recurrent) + 11 DSA (sparse MLA), 288-expert MoE, mHC hyper-connections, vision tower |
| quantization | oQ4e mixed precision, affine, group size 64 base, per-module bit-width overrides |
| iMatrix | 667 modules calibrated from the model's own activations; the 59 MTP-head tensors are quantized without iMatrix |
| MTP head | preserved (stored upstream as layer 45, remapped to language_model.mtp.*) |
| vision | preserved, 347 tensors, verified with a real image request |
| runtime | oMLX 0.6.3 |
How it was made
- Abliterated BF16 weights from Blackfrost-Research (599 GB, 120 shards), derived from zai-org/GLM-5.3-Flash.
- iMatrix measured from the model's own activations on a disk-built calibration proxy (the source exceeds the in-memory calibration limit).
- Body quantized with iMatrix-calibrated oQ4e; the MTP head preserved through conversion with a glm5_next-specific sanitize patch (stock oMLX 0.6.3 drops it) and quantized alongside the body.
- Vision tower carried through and verified on a real image, not a loader check. Coherence, refusal spot-checks, and speed all verified by reading actual outputs.
Limitations
- Capability benchmarks have not been run; conversion correctness and throughput were verified.
- Refusal behaviour was not systematically re-measured on this build.
- With DFlash2 on the stock adapter, every request pays the full prompt prefill (~46 s at 20k tokens): the GLM adapter disables the prefix cache because GLM's hybrid cache state is not serializable there. Our prefix-cache patch adds that serialization (multi-turn TTFT 0.32 s measured at both temperatures; publishing to GitHub shortly, PR pending upstream). The hit is not yet guaranteed on every turn - one longer-generation run re-prefilled instead - so MTP remains the safer choice for repeated long-context chat until that is resolved.
- Speculative output is greedy-equivalent but not bit-identical to plain decoding (rare floating-point tie-break forks; both continuations valid).
- No native thinking-off exists in the GLM-5.3 template.
Credits
- Z.ai (zai-org) for GLM-5.3-Flash.
- Blackfrost-Research for the abliterated BF16 weights this build was quantized from.
- inco.ai for the GLM-5.3-Flash-DFlash2 drafter.
- oMLX and the authors of PRs #3228 and #3261 for the GLM runtime and adapters.
Disclaimer and licence
This is an independent, unaffiliated conversion. It is not endorsed by, developed with, sponsored by, or associated with Z.ai, Blackfrost-Research, inco.ai, or oMLX. None of them reviewed, approved, or contributed to this repository, and none of them bear any responsibility for it.
It does not represent GLM-5.3-Flash. The behavior of this model is not the behavior of the official release: refusal behavior has been removed by a third-party weight modification. Do not treat any output as representative of, or attributable to, the upstream model or any party named here. "GLM" is used solely to identify the base architecture.
Changes made to the upstream work: quantized to mixed-precision oQ4e with iMatrix calibration; MTP draft head preserved and quantized; vision tower preserved; no template or persona changes - the source checkpoint's template ships as-is; no weights modified beyond quantization (the abliteration itself is upstream work).
Licence: MIT, inherited from the upstream model.
LocalMaxxing