Qwen3.8-27B-Uncensored-OptiQ-4bit (MLX)

A sensitivity-aware mixed 4/8-bit OptiQ MLX quant of orcarouter/Qwen3.8-27B-Uncensored, the abliterated (refusal-removed) BF16 build of Qwen's Qwen/Qwen3.8-27B.

Unlike builds that reuse the base Qwen sensitivity ranking, this quant takes its per-layer KL-divergence sensitivity directly from the Orca uncensored weights against the BF16 source (not a uniform-4-bit reference), then allocates 8-bit to the sensitive layers and 4-bit to the rest. The native vision tower and MTP head are preserved.

• ~20.1 GB on disk, 5.13 bits/weight (target 5.0)
• 497 quantized tensors: 220 @ 8-bit, 277 @ 4-bit (group size 64)
• Vision tower kept at BF16 in a sidecar (optiq/optiq_vision.safetensors) for OptiQ serve, plus stock-named model.visual.* in model-visual.safetensors for stock mlx-vlm / LM Studio compat (same 333 tensors, bit-exact; sidecar retained so OptiQ vision path is unchanged)
• MTP head preserved as a sidecar (optiq/mtp.safetensors) for speculative decoding
• 262,144-token context, hybrid attention (48 linear + 16 full attention layers)

Benchmark — Capability Score 91.69 (fair)

Full 6-benchmark OptiQ suite, same Mac (M3 Ultra 96 GB), greedy decode. Five benchmarks use reasoning mode; HashHop uses the fair non-reasoning configuration (see note below).

For context, the reference mlx-community/Qwen3.8-27B-OptiQ-4bit reports 87.98.

KL divergence vs the BF16 source (64 prompts × 256 tokens): mean 0.101, median 0.008, p95 0.300.

HashHop note (reasoning-mode artifact)

With --reasoning the suite reports HashHop 74.0 (hops 4 / 100 / 96 / 96). The hops=1 collapse is an evaluation/setting interaction, not a capability gap: in thinking mode with a 4096-token budget the model degenerates into transcribing the dictionary and never emits the final answer. In non-reasoning (concise) mode the same quant answers hops=1 correctly in ~10 tokens (100%). Non-reasoning hops=4 is harder (68%), so the fair single-config HashHop is 91.0. For long repetitive-context extraction, prefer :no-think / concise mode.

Quantization recipe

optiq convert orcarouter/Qwen3.8-27B-Uncensored \
  --method optiq \
  --target-bpw 5.0 \
  --candidate-bits 4,8 \
  --group-size 64 \
  --reference bf16 \
  --calibration-mix optiq \
  --n-calibration 40

• mlx-optiq 0.5.19
• Reference: BF16 (the full BF16 source fits in 96 GB RAM, so sensitivities are measured against the true base rather than a 4-bit proxy)
• Calibration: OptiQ 6-domain mix, 40 samples

Optional: mixed-precision KV cache

A sensitivity-measured per-layer KV config is included in the source build notes and reproduces 5.00 average KV bits (12 layers @ 4-bit + 4 layers @ 8-bit across the 16 full-attention layers; linear-attention layers are skipped):

optiq kv-cache ./Qwen3.8-27B-Uncensored-OptiQ-4bit --target-bits 5.0 --candidate-bits 4,8
optiq serve --model ./Qwen3.8-27B-Uncensored-OptiQ-4bit --kv-config ./kv_config.json

Usage

Text (stock mlx-lm)

pip install mlx-lm

from mlx_lm import load, generate
model, tokenizer = load("j-llm/Qwen3.8-27B-Uncensored-OptiQ-4bit")
print(generate(model, tokenizer, "What is 12*13?", max_tokens=64))

Image + text (stock mlx-vlm)

pip install mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model, processor = load("j-llm/Qwen3.8-27B-Uncensored-OptiQ-4bit")
config = load_config("j-llm/Qwen3.8-27B-Uncensored-OptiQ-4bit")
prompt = apply_chat_template(processor, config, "Describe this image.", num_images=1)
print(generate(model, processor, prompt, image=["test.png"], max_tokens=256).text)

The vision tower lives in the optiq/ sidecar and is restored automatically by both OptiQ and current mlx-vlm. preprocessor_config.json is included so library and app scanners (e.g. MLXBar) detect the model as image+text.

OptiQ serving (OpenAI + Anthropic API, MTP, KV)

pip install mlx-optiq
optiq serve --model j-llm/Qwen3.8-27B-Uncensored-OptiQ-4bit --mtp

Serving note: optiq serve's vision path currently requires mlx-lm==0.31.x (_serve_single(self, request)); mlx-lm 0.32 changed that internal signature and breaks the vision patch. Pin mlx-lm<0.32 in the serving environment.

Model details

LM Studio

Built for LM Studio parity with stock Qwen/Qwen3.8-27B: embedded chat_template (tool + <think> reasoning), stock-named model.visual.* vision weights, and image-text-to-text pipeline tag. If the in-app HF browser fails to fetch files (proxy cache), sideload instead:

hf download j-llm/Qwen3.8-27B-Uncensored-OptiQ-4bit --local-dir ./Qwen3.8-27B-Uncensored-OptiQ-4bit

then in LM Studio: My Models → Import and select the downloaded folder. Use concise / :no-think prompts for deterministic tasks; the reasoning trace can loop on transcription-style prompts (see HashHop note above).

Safety

This is an abliterated / uncensored model: safety refusals have been removed from the base weights. It is shared for research and red-teaming. Run it behind access controls and do not expose it publicly without moderation. The upstream orcarouter release carries the same warning.

Provenance

• Source: orcarouter/Qwen3.8-27B-Uncensored (BF16, 55.6 GB), revision 8cb32d72…
• Quantized with mlx-optiq 0.5.19 on Apple M3 Ultra (96 GB), BF16 reference, 40-sample calibration.