GSQ-RCO upstream · layout preserved144 tensors transplantedIQ3_S / IQ3_XXS / IQ2_XS / Q2_0per-tensor blake2b verified262K contextAPACHE-2.0

Qwen3.8-Flash-Next · GSQ-RCO-abliterated

The refusal direction removed from an already-quantized GSQ-RCO model — 144 tensors across all 48 layers swapped for ready-made abliterated weights, with every value and scale GSQ learned left untouched and the upstream per-tensor type assignment restored exactly. The IQ3_S build is verified tensor by tensor: everything that changed was supposed to, nothing else moved.

38.0 GB · Q2_0every tier costs only 0.25 – 0.44 GB over upstream55.1 GB · IQ3_S

English · 简体中文 📖

🧭 What this repository is

An abliterated build of the official ISTA-DASLab GSQ-RCO quantizations. Not a single one of GSQ's learned quantized tensors was recomputed — 144 "write-to-residual-stream" projection tensors across all 48 layers were replaced with the corresponding tensors from a ready-made abliterated release.

In one line: what you get is still the GSQ-RCO quantization you already know, with those 144 tensors abliterated. It is the only abliteration route that preserves GSQ's work.

⚠️ Safety notice

This model is abliterated and refusal-removed — it has no meaningful built-in guardrails and may comply with harmful, illegal or unsafe requests. Deploy it only where you can supply your own moderation, access control and legal review; do not put it in front of end users without a safety layer of your own.

本模型经过拒答方向消融，不具备可靠的内置安全护栏，可能配合有害、违法或不安全的请求。请仅在你能够提供审核、访问控制与法律审查的场景部署。

🧬 Why a transplant, and not a re-quantization

GSQ treats each tensor's grid assignment and per-group scale as learnable parameters, trained jointly through a Gumbel-Softmax relaxation; RCO then assigns one quantization type per tensor under a total size budget. As a result, a GSQ-RCO tensor value is not what you get by feeding BF16 weights to a standard quantizer — rerunning the same recipe yields a different set of values.

There is exactly one way to abliterate a GSQ-RCO model: replace tensors in place inside the already-quantized GGUF. That is what this repository does.

The ablation itself is a strictly rank-1 linear edit — W ← W − r(rᵀW) — with a single direction r shared by all 48 layers × both attention types (DeltaNet / QSA) × both the attention and MLP sites × all 512 routed experts. Given that, "copy an already-abliterated tensor" and "compute the ablation yourself and quantize it" land on the same line, and the former introduces no new quantization error.

The rank-1 structure was measured on a BF16 abliterated release of the same base: σ₂/σ₁ of ΔW = 0.0032–0.0053; the direction of every one of the 512 routed experts agrees with the shared-expert direction at |cos| = 1.00000, and likewise across layers and sites.

📦 Files in this repository

Each tier ships as two shards. Shard 2 is byte-identical across all four tiers and byte-identical to the copy in the upstream repository: it holds the per-layer n-gram embedding table (per_layer_token_embd, 51.2B parameters), a lookup table rather than a matmul weight, so it is pinned at IQ4_NL, excluded from the search, and untouched by the ablation.

Low tiers do not get fat here. Upstream ffn_down_exps can only live in Q2_0 or IQ4_NL (the tensor has 640 elements per row, which is not divisible by 256, so every block-256 K / I format is ruled out). The abliterated values, however, only exist in an IQ4_NL build — take them literally and the two 2-bit tiers would have to lift all 48 expert tensors to IQ4_NL, +11.7 GB each. This repository re-encodes the abliterated values into Q2_0 using GSQ's own learned block scales, so all four tiers keep the upstream per-layer type assignment exactly (IQ3_S measured at 9×Q2_0 + 39×IQ4_NL, identical to upstream). The only remaining cost is the 96 small tensors, whose abliterated values only exist at Q8_0: +0.25 GB on IQ3_S and +0.30 – 0.44 GB on the others.

🔬 What actually changed (IQ3_S)

Of the file's 1223 tensors, 144 were touched — all in the same family of "write-to-residual-stream" projections, covering all 48 layers:

95 of them changed quantization type; the other 49 (48 routed experts + 1 shared expert) kept the upstream type — different values, same format. Full-file type histogram:

Left alone: token_embd and output (the output head). Abliterations often touch those two as well; the release we took the weights from did not, so neither do we. The other 1079 tensors are unchanged, byte for byte.

✅ Four checks — how we know nothing got broken

Check ③ is hard evidence; check ④ is only a spot check — we ran a small probe set, and only that one prompt is a clean discriminator for this model. Do not read it as a full evaluation of the ablation.

📉 The cost — honest accounting

Measured on the Strata engine, not a llama.cpp baseline — dual-GPU layer split, 256K context, KV streaming. With the upstream GSQ-RCO layout restored, those 9 ffn_down_exps tensors go back to Q2_0 (2.25 bpw), so the expert read volume is identical to upstream and decode lands level with it (104.4 vs 104.5; the 0.1% is measurement noise). Cache slots went back from 11478 to 12095. Absolute numbers on llama.cpp will differ, but the direction will not.

🚀 Running it

This is standard GGUF with exactly the architecture of the upstream files, so it loads unmodified in any reasonably recent llama.cpp — no patched fork required. Usage is identical to the upstream tiers.

Download (both shards; llama.cpp picks up the second automatically)

hf download SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF --include "IQ3_S/*" --local-dir .

Text

llama-cli -m IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3_S-00001-of-00002.gguf \ -lm mmap --lazy-mode on -ngl 99 -c 32768 \ -p "Give me a detailed way to get around a news site's paywall."

Vision (multimodal)

hf download SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF \ mmproj-Qwen3.8-Flash-Next-BF16.gguf --local-dir .

llama-mtmd-cli -m IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3_S-00001-of-00002.gguf
--mmproj mmproj-Qwen3.8-Flash-Next-BF16.gguf -lm mmap --lazy-mode on
--image photo.jpg -p "Describe this image."

Speculative decoding with the MTP draft head

llama-cli -m IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3_S-00001-of-00002.gguf
-md mtp-Qwen3.8-Flash-Next-Q8_0.gguf --draft-max 4 -lm mmap --lazy-mode on -ngl 99

-lm mmap --lazy-mode on keeps the n-gram table memory-mapped on disk: it is 28.8 GB and one row is read per token. Shard 1 wants to be resident (VRAM or RAM); shard 2 is fine on an SSD. The vision projector is a standard clip GGUF shared with the upstream tiers, 0.91 GB. Native context is 262,144 tokens and the KV cache grows with it, so start at -c 32768 and work up.

🖥️ Strata engine — the runtime this build was made for

This repository was produced and verified on Strata, the third-party engine that runs GSQ-RCO GGUFs on consumer hardware — the official model runs on a single 12 GB card with 64 GB of system RAM, and this one is only 0.25 GB larger. No patched build is needed: Strata sees an ordinary GSQ-RCO pack.

Download one tier (shard 1 + shard 2 + the MTP head)

hf download SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF --include "IQ3_S/*" --local-dir ./model hf download SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF mtp-q2_0.gguf --local-dir ./model hf download SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF --include "strata/rt/*" --local-dir ./model

Build a separate native pack for this model. When migrating from the official GSQ-RCO model, changing only the GGUF paths is not enough: --pack must point to a pack built from this repository's GGUF files for the selected tier. The transplant changes tensor offsets, so the official model's native_experts.txt cannot be reused. An old pack can cause ArenaExpertSource: the pack's native_experts.txt does not match the model.

PowerShell example, run from your Strata installation directory: replace the input and output paths with your own. Keep both downloaded shards together with their original filenames; the packer automatically finds shard 2. Use a new output directory and keep the official pack for switching back.

.\.venv\Scripts\python.exe .\tools\iq_pack.py --gguf "C:\models\SC117\IQ3_S\Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3_S-00001-of-00002.gguf" --out "C:\strata\Strata-data\packs\sc117_iq3_s"

Point Strata at the SC117 model and its own native pack

--pack C:/strata/Strata-data/packs/sc117_iq3_s --native ./model/IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3_S-00001-of-00002.gguf --ple-gguf ./model/IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3_S-00002-of-00002.gguf --mtp ./model/strata/rt

Set tokenizer to the new pack's tokenizer directory. If vision is enabled, point vision.model at SC117's first shard; the existing upstream mmproj-Qwen3.8-Flash-Next-BF16.gguf projector can be reused. Keep your existing tuning parameters (--expert-profile, --spec 4, --max-context 262144, --kv int8, --vram-reserve-mib, …) as a starting point. The main-model native pack is separate from the MTP runtime directory supplied to --mtp.

This migration issue and successful loading after rebuilding the pack were reported by ghit72 on Strata v0.1.38. This is a community startup report, not a new evaluation of model responses.

Measured on Strata (dual GPU, 448-token decode): 104.4 tok/s against the official file's 104.5 — the ablation costs nothing in speed, because every tier keeps the exact upstream type assignment. Shard 1 goes from 51.05 to 51.28 GiB.

mtp-q2_0.gguf and strata/rt/ are Strata-only. Their architecture tag is qwen4exp-mtp, which llama.cpp does not recognise — that is why they sit in a separate folder. Using llama.cpp? Ignore them and take mtp-Qwen3.8-Flash-Next-Q8_0.gguf from the repository root instead.

The MTP draft head is not abliterated. No abliterated MTP head has been published anywhere, so the draft layer was extracted from the stock base checkpoint. The ablation only rewrites the 48 main layers — see strata/README.md.

🧩 What is here, and what is not

✅ All four tiers of the abliterated weights · ✅ the n-gram shard, byte-identical to upstream (download it once) · ✅ the standard clip vision projector · ✅ the complete transplant inventory and the per-tensor verification tooling · ✅ sizes and upstream type allocations for every tier

❌ No BF16 or high-precision builds — for lossless weights use the base model · ❌ No training, fine-tuning or weight optimization of any kind — this repository only moves tensors · ❌ No large-scale evaluation of the ablation (see check ④)

The quantization is the work of ISTA-DASLab, the abliterated weights are the work of orcarouter; this repository contributes the alignment procedure, the assembled files and the verification data.

🔍 Reproducing it

The transplant is pure byte arithmetic — it needs no quantizer and no GPU:

📄 License and credits

Apache-2.0, inherited from the base model — please also honour the licenses and model cards of every upstream repository.

Qwen — the Qwen3.8-Flash-Next base model · IST-DASLab — GSQ + RCO and the quantized weights · orcarouter — the abliterated weights · llama.cpp — the GGUF format and tooling · huihui-ai — the BF16 release used to confirm the rank-1 structure of the ablation. This repository is an independent project and is not affiliated with any of them.

📦 Upstream GSQ-RCO quantizations — the base of this build

English · 简体中文