Qwen3.8 27B

MLX affine 4-bit · OrcaRouter · Measured configuration

Historical 202-question screen: 80.64 / 100 Mango weighted score · 23.98 min matched workload · 12/12 mechanical agent passes

The exact 4-bit subfolder of OrcaRouter's Qwen3.8-27B-Uncensored-MLX release. Mango benchmarked this MLX-packed target through TensorFold CUDA with the separately preserved DFlash2 draft model.

Benchmark results · Files · Download · Runtime · Provenance · Full collection

Join Infinity on Discord

Get model support, request models and quants, discuss benchmarks, and follow new releases and model improvements.

Join the Infinity community

All model tests, per-question scores, experiment history and scoring methods

Qwen medium versus xhigh

A separate matched experiment used these same MLX4 weights with TensorFold 1.1.6 + DFlash2, fixed 64K context and official thinking sampling: 404 collected questions, 403 valid questions graded, 12 mechanical agent cases and 22 additional open-ended responses per effort.

Xhigh minus medium: +1.54 points, paired 95% interval [-1.71, +4.82], 10,000 task-stratified resamples, seed 41. This sample does not establish a clear quality gain. Xhigh took 4.46× as long on the fresh matched questions. Medium remains the practical choice under the existing economics rule; no final fleet/router or live mapping was changed.

Grading correction: one frozen IFEval annotation specifies # as a letter. The pinned reference checker substitutes a random alphabetic letter, so that case is unscored for both efforts. The collected prompts, answers, frozen suite and checker remain unchanged. Valid strict IFEval coverage is 39 prompts; language-detection seed is fixed at 0. All 200 historical strict IFEval grades and all 78 valid full-cohort strict grades retain their saved values. These corrected automatic scores supersede the earlier 82.88/84.29 report; the historical 202-question leaderboard and the frozen subjective ratings are unchanged.

Medium reused 202 earlier answers and 12 agent cases, so its full recorded workload spans older host/co-resident conditions. The fresh-202 timing control excludes reused cases. The historical 80.64/100 score and cross-model table below use the separate 202-question screen and must not be compared directly with these full-404 scores as a model change.

Blind subjective rating — two independent graders

Medium and Xhigh columns are the equal-weight mean of two blind graders. Claude and Codex each scored all 44 anonymized responses and froze their scores before the identity key was opened; neither saw the other's scores first. Both planned graders are complete; the panel consists of Claude and Codex. These ratings are separate from the automatic score.

Paired xhigh minus medium on the same 22 prompts: +0.41 [+0.05, +0.75] two-grader mean; Claude alone +0.09 [-0.41, +0.55], Codex alone +0.73 [+0.45, +1.00] (10,000 task-stratified resamples, seed 41). Graders differed by 0.64 points on average and gave identical scores to 24 of 44 responses. Both found xhigh clearly stronger on grounded inference and medium slightly stronger on creative writing and source faithfulness; the overall subjective edge for xhigh is small and depends on the grader.

Download automatic report · Download subjective coverage and aggregates

Preserved and documented by isilicon / Mango. The original model, modification and quantization remain the upstream creators' work. This repository adds attributed preservation and exact-configuration evaluation, with the selected source bytes unchanged.

At a glance

GB is decimal and GiB is binary. Payload size is disk/storage size, not a claim that the model fits in the same amount of VRAM. Runtime, KV cache, working memory and companions add their own requirements.

Mango evaluation

Fastest matched workload in this cohort: 23.98 minutes for the 202 questions plus 12 agent cases. It passed 19/20 strict instruction-following prompts and all 12 mechanical agent cases.

Pure EQ is 83.81, while theory-of-mind labeling is 55.56; averaging those tasks gives the lower 69.68 social/EQ domain. Its 80.64 overall score trails the Flash Next point estimates, with wide paired uncertainty.

Domain scores equally average their component task means. The overall score applies the weights above. EQ is continuous; the 12-case mechanical agent pass rate is only one component of the agentic domain.

Task-level quality and uncertainty

These are small released subsets and local graders. HumanEval/BFCL scores are not the complete official benchmark pipelines. Agent checks cover completion, evidence, permissions and applicable pricing totals; other answer semantics remain unjudged.

Recorded performance

Workload time excludes loading, restarts, wipe downtime and idle gaps. Latency/TTFT summaries cover suite requests only. The reported token rate divides engine usage counts by request time; it is not native decode throughput and token accounting differs across engines. No concurrent-serving result is implied.

Matched cohort comparison

The old mixed GLM row is a historical, internally capped baseline whose exact weights are unavailable. Other rows also differ in quant, engine, context and effort. Point ranks do not settle the quality ordering.

Paired score differences

Intervals use 10,000 paired task-stratified bootstrap resamples, seed 41, with fixed task/domain weights. An interval containing zero does not establish a quality winner. See methodology for scope and definitions.

Download the current full numerical report and methodology. The original compact evaluation is retained as the earlier preservation snapshot; its model score is unchanged, while the current report includes the expanded cohort. The numerical report contains aggregate statistics and hashes; no prompts or generated answers are published.

Runtime and compatibility

Keep the 4-bit/ subdirectory intact. The measured setup used TensorFold 1.1.6 and DFlash2. The target can be used in other compatible runtimes, but their throughput and quality settings require their own checks.

Original matched Qwen suite verified at 65,536 tokens; native medium effort. Current serving defaults may differ from that historical cohort.

The local evaluation covers text/tool tasks. It does not establish the full advertised context ceiling, image/audio quality or every optional speculative-decoding path. Publisher capabilities and Mango-verified behavior are separate scopes.

Companion artifacts

The selected payloads of this model and its linked companions total 19.93 GB before runtime/cache files. That is a download/storage total, not a RAM/VRAM fit guarantee.

Files and integrity

Full paths, byte sizes and complete SHA-256/Git blob identities are in SOURCE_MANIFEST.json. Available original README/license/notice files are also preserved. MIRROR_STATUS.json records the earlier source-file verification; MODEL_CARD_STATUS.json records this documentation refresh separately.

Download and use

The commands pin the already-verified mirror weight revision. Documentation can advance on main without changing that preserved weight snapshot. Keep original subfolders and all shards.

python -m pip install -U huggingface_hub

hf download isilicon/mango-qwen38-27b-uncensored-mlx4 \
  4-bit/chat_template.jinja \
  4-bit/config.json \
  4-bit/generation_config.json \
  4-bit/model-00001-of-00003.safetensors \
  4-bit/model-00002-of-00003.safetensors \
  4-bit/model-00003-of-00003.safetensors \
  4-bit/model.safetensors.index.json \
  4-bit/preprocessor_config.json \
  4-bit/processor_config.json \
  4-bit/tokenizer.json \
  4-bit/tokenizer_config.json \
  4-bit/video_preprocessor_config.json \
  4-bit/vocab.json \
  --revision 8254f65f91b2066ae8371aa9ddbb35149e16ff15 \
  --local-dir ./weights/qwen-mlx

hf download isilicon/mango-dflash2 \
  config.json \
  model.safetensors \
  --revision e383e96a1ac8822a9012a57d43c40d488993a7ad \
  --local-dir ./weights/dflash2

Load example and setup scope

tensorfold-native serve ./weights/qwen-mlx/4-bit \
  --name mango-qwen27 --host 127.0.0.1 --port 8080 \
  --context 65536 --parallel 1 --thinking --reasoning-effort medium \
  --drafter ./weights/dflash2 --no-update-check

This example uses the TensorFold 1.1.6 command family from the measured CUDA setup. Install that compatible engine separately. The source card also documents Apple MLX runtimes, whose performance is not measured here.

Provenance and license

The publisher describes affine MLX quantization with group size 64 and selected tensors retained at higher precision. The preserved format is MLX; the reported Mango engine was TensorFold on NVIDIA hardware, so these numbers are not mlx-vlm or Apple Silicon benchmarks.

Original notices, base-model conditions and the publisher's stated limitations still apply. Downloadable preservation does not add new license rights. Source-described refusal modification and publisher-side benchmark claims have not been independently re-measured by this Mango task cohort.

Original publisher documentation

The unchanged upstream README and pinned source card retain the original usage guidance, technical discussion, attribution, caveats and citations.

Read the complete original publisher card

Source reference. The following is the original publisher's documentation. Its other quants, full-model files, context claims and benchmark tables can exceed the selection in this mirror. Relative asset/file URLs are qualified to the pinned original source; publisher measurements remain separate from Mango measurements.

Qwen3.8-27B-Uncensored-MLX

An abliterated (refusal-removed) MLX build of Qwen's Qwen3.8-27B — 2 / 4 / 6 / 8-bit for Apple Silicon

One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.

Website · Model Catalog · Model API · GitHub · Discord · X

---

An abliterated (refusal-removed) build of Qwen/Qwen3.8-27B — a 27B-parameter dense, hybrid-attention (Gated DeltaNet linear + full attention) native vision-language model with thinking control, tool-calling and an MTP head — quantized to MLX format for Apple Silicon. Four precisions are provided — 2 / 4 / 6 / 8-bit (affine, group size 64) — each as a subfolder, with the 4-bit build also mirrored at the repo root so that orcarouter/Qwen3.8-27B-Uncensored-MLX loads directly in LM Studio and other tools that treat a repo as a single model. The vision tower, norms and conv layers are kept in BF16; only the language-model linear weights (including embed_tokens / lm_head) are quantized. Browse all models in the OrcaRouter Model Catalog. This model is deployed as API here.

---

⚠️ Disclaimer & risks — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). As a direct consequence:

• It will comply with harmful, unethical, offensive, or illegal requests that the original Qwen3.8-27B would refuse. It has no meaningful built-in guardrails.
• It is released strictly for legitimate research — interpretability, AI-safety and refusal-mechanism study, red-teaming, robustness evaluation, and controlled experiments.
• You assume full responsibility and liability for how you use it and for everything it generates. Add your own safety, moderation and abuse-prevention layers before any deployment.
• Use must comply with the Apache 2.0 License inherited from the base model, and all laws and regulations that apply to you.
• The authors and uploaders accept no liability for any misuse or harm. Outputs do not reflect the views of the uploaders or of Qwen / Alibaba.

Specific risks

• Harmful content on demand — it will produce instructions for malware, exploits, weapons, fraud and other illegal or dangerous activity when asked.
• No refusals — jailbreak / safety probes "succeed" trivially; do not mistake this for a passing safety evaluation.
• Confident falsehoods & bias — it can generate false, defamatory, biased or offensive text and present it authoritatively.
• Expanded attack surface — preserved vision, tool-calling and 262K context mean these risks extend to image understanding and autonomous / agentic use.
• Quantization noise — lower-bit builds (esp. 2-bit) add instability on top of the above; outputs can be degraded or nonsensical.

Intended use vs out of scope

• Intended: AI-safety and interpretability research, refusal-mechanism study, red-teaming, guardrail and robustness evaluation, controlled academic experiments.
• Out of scope: any deployment to end users, minors, or production without your own moderation / safety layer; any unlawful, harmful, or rights-infringing use.

By downloading or using this model you acknowledge and accept the above.

---

Available quantizations

2-bit warning: at 27B, 2-bit quantization collapses generation quality (repetition loops, garbled output). It is included only as an extreme-compression archive; do not use it for real work — prefer 4-bit or higher.

Repo root = 4-bit/. The root of this repo holds a copy of the 4-bit build, so --model orcarouter/Qwen3.8-27B-Uncensored-MLX (no subfolder) resolves to 4-bit. Use the subfolder paths to pick any other precision.

---

Verification & test results

All builds were quantized from the same abliterated BF16 source and verified numerically (dequantized weights vs. source) plus tested by generation on GPU.

• Uncensored preserved: red-team probes (exploit walkthrough, controversial argument) return substantive content with zero refusals on 4 / 6 / 8-bit.
• Multimodal preserved: shapes, colors, position, background and text in a probe image are described correctly on 4 / 6 / 8-bit.
• Speed: ~32–37 tok/s steady-state on a single H200 (MLX CUDA backend). MLX's native target is Apple Silicon (Metal).

Note: on 6-bit, mlx's offline mx.dequantize mis-unpacks these weights (a library edge case), so correctness is verified by clean generation — inference is unaffected.

---

Usage (mlx-vlm, Apple Silicon)

pip install -U mlx-vlm    # needs mlx-vlm >= 0.6.13, mlx >= 0.32

# download one precision (e.g. 4-bit) from the subfolder
hf download orcarouter/Qwen3.8-27B-Uncensored-MLX --include "4-bit/*" \
    --local-dir ./Qwen3.8-27B-Uncensored-MLX

# text
python -m mlx_vlm generate \
    --model ./Qwen3.8-27B-Uncensored-MLX/4-bit \
    --prompt "Explain quantum entanglement in one sentence." --max-tokens 256

# vision (image + text)
python -m mlx_vlm generate \
    --model ./Qwen3.8-27B-Uncensored-MLX/4-bit \
    --image path/to/image.png \
    --prompt "Describe this image." --max-tokens 256

# OpenAI-compatible server
python -m mlx_vlm server --model ./Qwen3.8-27B-Uncensored-MLX/4-bit --port 8080

On Apple Silicon the Metal backend is used automatically — no CUDA setup needed. (On a Linux CUDA backend, vision requires MLX_CUDA_USE_CUDNN_SDPA=0; this does not apply on macOS.)

---

Multi-Token Prediction (MTP) — speculative decoding

This model has a native MTP head. In MLX, MTP is loaded as a separate drafter for speculative decoding: the main model is loaded with the MTP weights stripped, and the drafter is passed explicitly. The drafter lives in the mtp/ subfolder of this repo (model_type: qwen3_5_mtp) and works with any main-model precision (4 / 6 / 8-bit).

Setting an mtp_enabled flag on the main model alone does nothing — MLX needs the separate drafter passed via --draft-model … --draft-kind mtp.

# fetch a main-model precision (e.g. 6-bit) plus the MTP drafter
hf download orcarouter/Qwen3.8-27B-Uncensored-MLX --include "6-bit/*" "mtp/*" \
    --local-dir ./Qwen3.8-27B-Uncensored-MLX

# generate with MTP speculative decoding
python -m mlx_vlm generate \
    --model       ./Qwen3.8-27B-Uncensored-MLX/6-bit \
    --draft-model ./Qwen3.8-27B-Uncensored-MLX/mtp \
    --draft-kind mtp --draft-block-size 4 \
    --prompt "Explain quantum entanglement in one sentence." --max-tokens 256

# OpenAI-compatible server with MTP
python -m mlx_vlm server \
    --model       ./Qwen3.8-27B-Uncensored-MLX/6-bit \
    --draft-model ./Qwen3.8-27B-Uncensored-MLX/mtp \
    --draft-kind mtp --draft-block-size 4 --port 8080

Requirements: an mlx-vlm build with the qwen3_5_mtp drafter and --draft-kind mtp (available on mlx-vlm main). MTP acceptance is lossless — with greedy decoding the output is identical to running without the drafter, just fewer forward passes on accepted tokens. The speedup is realized on Apple Silicon (Metal); one drafter serves all precisions.

---

Usage (LM Studio)

Search for orcarouter/Qwen3.8-27B-Uncensored-MLX in LM Studio and download it — the repo root is the 4-bit build, and the other precisions appear as separate download options.

Three things to get right:

1. This repo is gated. LM Studio downloads anonymously by default and will get an HTTP
   401. Accept the terms on the model page once, then paste a Hugging Face read token into LM Studio under Settings → Integrations → Hugging Face.
2. Turn off KV cache quantization. MLX vision models do not support it on this architecture, and loading fails during initialization if it is enabled (mlx-engine#286).
3. Pick a quant that fits. 8-bit is ~29.5 GB on disk and wants a 64 GB Mac; 6-bit suits 48 GB; 4-bit (~16 GB) is the right choice on a 32 GB Mac. LM Studio's "Likely too large" badge is a RAM warning, not an error.

If you are on an older LM Studio MLX runtime, update it (Settings → Runtime): qwen3_5 support landed in mlx-vlm 0.6.x, and older runtimes cannot load this architecture at all.

---

Model details

Preservation record

Selected source bytes are verified against pinned metadata; the original eight graded mirrors also passed the previously recorded anonymous 1 MiB weight-prefix checks. This is not a full independent-device restore/load test. Hugging Face repository copies share a provider and do not guarantee permanent storage.

Collection · Source manifest · Original documentation