Qwen3.8 27B Cyber

IQ4_XS imatrix · embedded MTP · Measured configuration

70.27 / 100 Mango weighted score · 29.74 min matched workload · 12/12 mechanical agent passes

cyjin-yl's IQ4_XS agentic-imatrix package of philbert440's Qwen3.8 27B Uncensored Cyber, selecting the file with the grafted MTP head.

Benchmark results · Files · Download · Runtime · Provenance · Full collection

Join Infinity on Discord

Get model support, request models and quants, discuss benchmarks, and follow new releases and model improvements.

Join the Infinity community

All model tests, per-question scores, experiment history and scoring methods

Qwen effort study — separate full-404 cohort: medium 82.87/100 versus xhigh 84.41/100; paired difference +1.54 points (95% interval [-1.71, +4.82]). 404 collected questions, 403 valid questions graded per effort; one invalid reference annotation excluded symmetrically. The historical 202-question cross-model comparison below is unchanged. Study, timing and blind subjective results.

Preserved and documented by isilicon / Mango. The original model, modification and quantization remain the upstream creators' work. This repository adds attributed preservation and exact-configuration evaluation, with the selected source bytes unchanged.

At a glance

GB is decimal and GiB is binary. Payload size is disk/storage size, not a claim that the model fits in the same amount of VRAM. Runtime, KV cache, working memory and companions add their own requirements.

Mango evaluation

This exact configuration scored 70.27/100 in 29.74 workload minutes, with zero infrastructure errors or length finishes. Local coding 17/20, function calls 28/40, instructions 11/20, mechanical agents 12/12.

Social/EQ 43.83 combines pure EQ 37.65 and OpenToM 9/18. Three of 30 EQ outputs failed parsing/range checks and four native formula values were negative; no clipping was applied. The score drop is not entirely a formatting failure.

It trails general Qwen MLX4 by 10.37 points, with a paired 95% interval [-15.29,-5.42], and previous Flash by 13.92[-19.09,-8.75]. Its specialist label does not establish an upgrade on this sample; no promotion is supported. Different quant, lineage, draft and runtime settings prevent a causal quantization claim.

Domain scores equally average their component task means. The overall score applies the weights above. EQ is continuous; the 12-case mechanical agent pass rate is only one component of the agentic domain.

Task-level quality and uncertainty

These are small released subsets and local graders. HumanEval/BFCL scores are not the complete official benchmark pipelines. Agent checks cover completion, evidence, permissions and applicable pricing totals; other answer semantics remain unjudged.

Recorded performance

Workload time excludes loading, restarts, wipe downtime and idle gaps. Latency/TTFT summaries cover suite requests only. The reported token rate divides engine usage counts by request time; it is not native decode throughput and token accounting differs across engines. No concurrent-serving result is implied.

Matched cohort comparison

The old mixed GLM row is a historical, internally capped baseline whose exact weights are unavailable. Other rows also differ in quant, engine, context and effort. Point ranks do not settle the quality ordering.

Paired score differences

Intervals use 10,000 paired task-stratified bootstrap resamples, seed 41, with fixed task/domain weights. An interval containing zero does not establish a quality winner. See methodology for scope and definitions.

Download the current full numerical report and methodology. The numerical report contains aggregate statistics and hashes; no prompts or generated answers are published.

Runtime and compatibility

Mango tested this exact embedded/grafted MTP file through its compatible native llama.cpp path. The source vision projector is outside the selected file; image input is not established by the text benchmark.

202 matched questions plus 12 mechanical cases; configured initial 32768 context; native medium (API high mapping). Native effort headers verified 202/202; context headers were absent, so profile/preflight receipts establish the configured context.

The local evaluation covers text/tool tasks. It does not establish the full advertised context ceiling, image/audio quality or every optional speculative-decoding path. Publisher capabilities and Mango-verified behavior are separate scopes.

Companion artifacts

No separate companion mirror is declared in this selected manifest. This does not imply that every optional upstream projector, draft head or multimodal path is included; consult the exact file list.

Files and integrity

Full paths, byte sizes and complete SHA-256/Git blob identities are in SOURCE_MANIFEST.json. Available original README/license/notice files are also preserved. MIRROR_STATUS.json records the earlier source-file verification; MODEL_CARD_STATUS.json records this documentation refresh separately.

Download and use

The commands pin the already-verified mirror weight revision. Documentation can advance on main without changing that preserved weight snapshot. Keep original subfolders and all shards.

python -m pip install -U huggingface_hub

hf download isilicon/mango-qwen27-cyber-iq4xs-mtp \
  Qwen3.8-27B-Uncensored-Cyber-IQ4_XS-imatrix-fromq8-plus-mtp.gguf \
  --revision 353675ec5767ecbded9cbcabce7da16c0258d0c4 \
  --local-dir ./weights/qwen27-cyber-iq4xs-mtp

Load example and setup scope

llama-server \
  --model ./weights/qwen27-cyber-iq4xs-mtp/Qwen3.8-27B-Uncensored-Cyber-IQ4_XS-imatrix-fromq8-plus-mtp.gguf \
  --host 127.0.0.1 --port 8080 --ctx-size 32768 \
  --jinja --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0

This is a minimal text-serving example for a compatible build, not a replay of every benchmark fitting/offload flag. The recorded configuration and engine caveats define the tested scope.

Provenance and license

The source explicitly attributes behavior and model weights to philbert440; its contribution is quantization and calibration on agentic-coding traffic. The publisher's V100/FastLLM measurements concern that environment and are not Mango RTX PRO 6000 results.

Original notices, base-model conditions and the publisher's stated limitations still apply. Downloadable preservation does not add new license rights. Source-described refusal modification and publisher-side benchmark claims have not been independently re-measured by this Mango task cohort.

Original publisher documentation

The unchanged upstream README and pinned source card retain the original usage guidance, technical discussion, attribution, caveats and citations.

Read the complete original publisher card

Source reference. The following is the original publisher's documentation. Its other quants, full-model files, context claims and benchmark tables can exceed the selection in this mirror. Relative asset/file URLs are qualified to the pinned original source; publisher measurements remain separate from Mango measurements.

Qwen3.8-27B-Uncensored-Cyber — GGUF quantized with an agentic imatrix

GGUF quantizations of philbert440/Qwen3.8-27B-Uncensored-Cyber, calibrated on an importance matrix built from real agentic-coding traffic rather than generic English prose.

All model weights and behaviour are philbert440's work. This repository contributes only the quantization and the calibration methodology — no fine-tuning, no merging, no behavioural change of any kind was performed here.

Inherited from upstream, and stated plainly so nobody is surprised: this is a de-refusal ("abliterated") model specialized for the cyber/offensive-security domain. The vision tower and the MTP speculative-decoding head are preserved upstream, which is what makes the multimodal path and the MTP graft work at all. Read the base model's card before using it — it documents the recipe and the evaluation.

Status (2026-08-20): the imatrix and both IQ4_XS artifacts are uploaded. Sizes and SHA-256 values below are the Hugging Face LFS objects, checked against the local production artifact for the MTP build.

Files

The two model files have identical IQ4_XS/imatrix text weights. The plus-mtp file adds the MTP tensors from the matching upstream model; use it only with a runtime that recognizes that graft. For a conventional GGUF loader, use the non-MTP file unless its MTP support has been verified. The Q5_K_M comparison below was measured locally and is not published in this repository.

Why a different calibration corpus

The usual llama.cpp calibration sets (wiki.train.raw, groups_merged.txt) are generic prose. The tokens that actually break under quantization in agentic use are a different population:

• tool-call JSON and tool names (bash, read, web_search)
• chat-template special tokens (<|im_start|>, <tool_call>)
• long literals that must be reproduced verbatim — repository paths, package names, commit hashes, device UUIDs
• mixed Chinese/English (prose in Chinese, paths and code in English)

Those barely occur in generic prose, so a prose-derived imatrix treats their channels as unimportant and the quantizer spends its error budget there first. The concrete failure this targets: an agent writing /home/eze/Documents/PotouI for /home/ezra/Documents/Proto-UI, then reading its own mangled output back and concluding its context is inconsistent.

Calibration corpus

Extracted from real agent session logs (an agent CLI driving coding tasks against this model) — not synthetic prompts:

The corpus itself is not published. It is private working data — real repository paths, real shell commands, real conversations. The resulting imatrix-agentic-v2.gguf is published so the method is reproducible; rebuild your own corpus from your own traffic to reproduce it end to end.

How it was made

# importance matrix, computed on the Q8_0 release
llama-imatrix -m Qwen3.8-27B-Uncensored-Cyber-Q8_0.gguf \
              -f calib-agentic.txt \
              -o imatrix-agentic-v2.gguf \
              -ngl 99 -c 512 --parse-special

# quantization (IQ4_XS shown; Q5_K_M identical apart from the type)
llama-quantize --allow-requantize --imatrix imatrix-agentic-v2.gguf \
               --output-tensor-type q8_0 --token-embedding-type q8_0 \
               Qwen3.8-27B-Uncensored-Cyber-Q8_0.gguf out.gguf IQ4_XS

Three choices worth explaining:

--parse-special is not optional here. The corpus is rendered with the model's own chat-template markers. Without that flag llama.cpp tokenizes <|im_start|> as literal text — <, |, im, _start, … — so the special tokens never enter the statistics at all, which defeats the point of building this corpus. Measured effect on the same corpus family: 0.342 → 0.299 tokens per byte (−12.7%), consistent with those markers collapsing into single tokens.

Context stayed at 512, deliberately. The intuitive move is to match the 262 K context this model is served at. Community testing says the opposite: for a fixed token budget a small context usually beats 4096, because it yields more and more-diverse samples and a better-conditioned estimate. 512 is also llama.cpp's long-standing default.

--output-tensor-type q8_0 --token-embedding-type q8_0. Output-layer quantization error turns directly into picking the wrong token, which is precisely the verbatim-copy failure above. Costs roughly 1.5 GB.

Honest caveats

• Quantized from the Q8_0 release, not from BF16 (--allow-requantize). Q8_0 is near-lossless and the error is dominated by the Q5/IQ4 step, but this is still a second quantization.
• Comparison scope is narrow. The measured numbers below use a held-out agentic corpus drawn from different sessions than calibration (verified disjoint). They test this deployment shape, not general writing, multilingual knowledge, or safety.
• imatrix helps most below Q5_K_M. Community consensus puts the large gains in the Q3/Q4 range. IQ4_XS is squarely in useful territory; Q5_K_M is near the upper edge where the effect gets modest.
• Calibration is domain-matched to agentic coding. For creative writing or general chat, a generic imatrix may suit you better.

Context

Built while making this model usable for agentic work on a single V100 32 GB (SM70) — FP16-only tensor cores, no INT8/FP8/FP4 TC, no cp.async. On that hardware decode is memory-bandwidth bound, so IQ4_XS (~4.25 bpw) reads about 26% fewer bytes per token than Q5_K_M (~5.33 bpw) and frees roughly 4 GB for KV cache — which is why it is the primary target here rather than the largest quant that fits.

The production configuration used for the measurements below is FastLLM on one V100: IQ4_XS-imatrix-fromq8-plus-mtp, turbo3 KV cache, batch 4, and a 262,144-token limit. Multimodal use also requires the matching Qwen3.8 vision projector; it is a separate artifact and is not embedded in either text-model GGUF here.

Lineage

Qwen3.8-27B                                      — Qwen team
  └─ philbert440/Qwen3.8-27B-Uncensored-Aggressive   α=1.15, recipe-v2 refusal ablation
       └─ philbert440/Qwen3.8-27B-Uncensored-Cyber   residual-cyber peel, β=1.0, apply_from=4
            └─ this repository                       quantization only

Quantized from philbert440's own Qwen3.8-27B-Uncensored-Cyber-Q8_0.gguf release.

Credits

• philbert440 — the Cyber model, its Aggressive base, and the Q8_0 GGUF these quantizations are derived from. Everything this repo is useful for is downstream of that work.
• Qwen team — Qwen3.8-27B, the underlying model.
• ggml-org/llama.cpp — llama-imatrix and llama-quantize. The importance-matrix approach itself comes from PR #4861; the GGUF imatrix format from PR #9400.
• The llama.cpp community discussions on calibration data and context size (#5006), which are why the calibration context here stayed at 512 instead of being naively enlarged.

License

apache-2.0, inherited from the base model. These are quantizations of philbert440's weights; the upstream terms govern.

---

实测数据(2026-08-20)

质量: 留出集配对困惑度

留出集与校准语料的 512 字节窗口重叠率 0.00% / 0.11%,是干净评测集。 三组用同一份留出集、同一个 --chunks 60。

不要按这两个置信区间去比。 两组评的是同一批分块,误差高度相关, 按独立区间读会严重低估判别力。做配对比较(逐块还原 NLL 后对差值做检验):

IQ4_XS vs Q5_K_M   ΔPPL = +0.417% ± 0.187%   t = 2.22   60 块中 39 块更差

也就是说:从 5.33 bpw 降到 4.25 bpw 的代价是 0.42%,差异真实、方向一致, 但幅度很小。配对把不确定度从 ±1.74% 压到 ±0.187%,约 9 倍。

功能: 走真实推理栈(fastllm on V100)的验收

"逐字抄写"测的是模型把上下文里的路径、UUID、commit 哈希原样复制的能力。 它比困惑度更贴近 agent 的真实失败模式:抄歪一个字符命令就失败,而困惑度 是全词表平均,对这种尾部行为几乎不敏感。

速度: 控制上下文长度后的配对结果

早期把不同上下文长度的请求直接比较,一度误判 IQ4_XS 比 Q5_K_M 慢。控制成 相同 token 形状后,8 组端到端配对样本全部同向:IQ4_XS 平均为 Q5_K_M 的 1.20x。这是单卡 V100 + FastLLM 的部署结果,不是跨推理栈的通用保证。

当前生产仍有明确的算子余量:decode 的主路径是通用 mmvq,而 SM70 IQ4_XS MMQ 只覆盖较大的 n;因此 1.20x 主要来自更低的权重带宽和当前融合路径, 不代表专用 decode kernel 已经用满硬件。

Preservation record

Selected source bytes are verified against pinned metadata; the original eight graded mirrors also passed the previously recorded anonymous 1 MiB weight-prefix checks. This is not a full independent-device restore/load test. Hugging Face repository copies share a provider and do not guarantee permanent storage.

Collection · Source manifest · Original documentation