Qwen3.5-4B-Zwyrol-KQuant-Uncensored (MLX, 4-bit)
The refusal removal of Zwyrol, redone on Nace's KQuant 4-bit pack instead of the plain mlx-community 4-bit file that Zwyrol used. It has the same size as KQuant (3.03 GB) and the same MLX format (affine, group 64, 4 bits), so it runs at the same speed. No training: the only change is a direct edit of 29 weight matrices.
It was made for Stefan, the author's private voice assistant. Stefan speaks Polish and calls tools (it reads the screen, checks agent status and keeps notes). The closeness and tool tests below use Stefan's system prompt and tool list.
Why KQuant
A 4-bit model stores each weight in 4 bits instead of 16, so the file is much smaller (3.03 GB here). In MLX, each group of 64 weights shares one scale and one offset, and these two numbers decide how much quality the rounding loses. The usual mlx-community packs take them from the smallest and largest weight in each group. Nace's KQuant packs choose them with a better (private) method, but keep the standard MLX format, so any MLX loader runs them at the same size and speed.
On Stefan's 154 Polish test prompts, unedited KQuant's distance to stock bf16 was about 29 % lower than the unedited mlx-community file's (KL 0.0785 against 0.1107, table below). Zwyrol was built on the mlx-community file. This model puts the same refusal edit on the better base.
Uncensored. This model answers requests the original refuses (6 of 104 harmful test prompts refused, against 103 for the original). You are responsible for how you use it and for any filtering your application needs.
What was changed
- Base: KQuant Q4. The 4-bit codes are plain rounding on KQuant's grid (checked: 100 % of codes match), so all of KQuant's gain is in its scales and offsets.
- Refusal removal (Heretic). The same 29 matrices as Zwyrol (
linear_attn.out_proj,self_attn.o_proj,mlp.down_proj, layers 14-31), from TheCluster's bf16 Heretic model, rounded on KQuant's own scales and offsets for those matrices (KQuant's quantizer is not public). Every other tensor is bit-for-bit KQuant, and the output layer stays tied to the embedding. List:ablation.json. - No tool-call gate. Zwyrol also changed one output row to make tool calls more likely. On KQuant the refusal edit alone already calls tools as often as Zwyrol does (tables below). A refitted gate was built and tested; by a rule fixed before the test it was not better, so it is left out: fewer edits, smaller file.
Why not copy Zwyrol's edits into KQuant? That was tried first. It keeps Zwyrol's whole output layer and its gate, which was fitted on another model. On this base the gate fires at the wrong times: in the app test below it called a tool on 16 of 18 questions that need none, and its first-token KL to the full model was the worst of all builds (1.10 against Heretic bf16; this model 0.30).
Results
All numbers come from runs of these exact files on one Mac Studio (M2 Max, 32 GB), greedy decoding, thinking off. Tool tests: Stefan's Polish system prompt with read-only tools (screen, agent status, notes).
Closeness to the full-precision models, 154 Polish prompts with Stefan's system prompt and 6 tools (mean KL over 64-token replies, lower is closer; reference generated by the bf16 model):
| model (4-bit) | vs Heretic bf16 (the edited full model) | vs Qwen3.5-4B bf16 (stock) |
|---|---|---|
| mlx-community 4-bit, unedited | 0.1346 | 0.1107 |
| KQuant, unedited | 0.0992 | 0.0785 |
| Zwyrol | 0.1227 | 0.2001 |
| this model | 0.0871 | 0.1344 |
This model is the closest 4-bit file to Heretic bf16. Against Zwyrol the difference is -0.035, 95 % CI [-0.052, -0.020] (paired bootstrap over prompts). Against stock bf16 it is farther than the unedited packs: that distance is the refusal edit itself.
Tool decisions (38 screen questions, 51 with an expected tool, 18 without a tool; the test set was never used to fit or choose anything):
| model | screen questions that call | expected tool | false calls |
|---|---|---|---|
| original mlx-community 4-bit (earlier run) | 13/38 | 23/51 | 0/18 |
| Zwyrol | 32/38 | 41/51 | 0/18 |
| this model | 35/38 | 41/51 | 2/18 |
Per question against Zwyrol: 4 screen questions won, 1 lost (sign test p = 0.38); 2 more false calls (p = 0.5). Neither difference is significant at this size. Held-out set (90 questions, written by an agent that never saw the test set): Zwyrol 25/31 screen, 33/45 expected tool, 1/45 false; this model 26/31, 33/45, 1/45.
In the Stefan app itself (its full system prompt, 6 tools and its tool-pick step; one run each, temperature 0; Zwyrol run twice with identical counts):
| model | screen calls | expected tool | false calls | fact checks | harmless instructions: obeyed / tool call instead (of 75) |
|---|---|---|---|---|---|
| Zwyrol | 34/38 | 37/51 | 1/18 | 6/6 | 45 / 28 |
| this model | 36/38 | 41/51 | 4/18 | 2/6 | 53 / 21 |
| refusals, 104 harmful test prompts | English | with "Odpowiedz po polsku." |
|---|---|---|
| original 4-bit | 103 | 100 |
| Zwyrol | 4 | 0 |
| this model | 6 | 1 |
Known limits
- More false tool calls than Zwyrol in the app: 4 of 18 no-tool questions against 1 (sign test p = 0.375). All four
are
screen_readorscreen_lookon general-knowledge questions. - Fact checks are mixed: 2 of 6 in the app (Zwyrol 6 of 6); 63 of 85 over the three test sets without the app (Zwyrol 58 of 85). Example misses: "Kto napisał Pana Tadeusza?" → "Juliusz Słowacki"; "17 × 3" → 45.
- Tested with one Polish system prompt and one tool list. Another prompt may change where it calls tools.
- The refusal count uses Heretic's keyword list (plus Polish keywords); it counts some soft refusals as answers.
Use
pip install mlx-lm
mlx_lm.generate --model Kaziko/Qwen3.5-4B-Zwyrol-KQuant-Uncensored --prompt "Cześć!"
It loads like the KQuant pack; the config is unchanged.
mtp.safetensors is the stock multi-token prediction (MTP) layer of
Qwen/Qwen3.5-4B (15 tensors, bf16, unchanged). The 4-bit MLX files do not
include it. A server that supports MTP drafts can use it to reply faster, for example
Locals Qwen Vision (set LOCALS_MTP to the file's path). Plain
mlx_lm ignores it. On this model
its first guess matched the model's next token on 82.8 % of 1,666 tokens (12 prompts, greedy). Greedy replies with
MTP drafts on were the same as with them off on 13 of 14 prompts; the one difference came at a near tie (logit gap
0.0156).
Credits
Qwen (base model, Apache-2.0) · Nace KQuant (4-bit pack, Apache-2.0) · p-e-w/heretic · coder3101/Qwen3.5-4B-heretic · TheCluster/Qwen3.5-4B-Heretic-MLX.