Qwen3.8 27B Abliterated · GSQ–Orca Hybrid

A compact, experimental GGUF with MTP included, built for local inference on smaller GPUs. It combines GSQ's quantized weights with selected abliterated weights from OrcaRouter and was developed and tested on an RTX 3060 12 GB.

This is my first open-source model release. My goal is to share something useful for local use, with a straightforward account of how it was built and tested. Abliteration—the weight modification used to reduce refusal behavior—comes from OrcaRouter.

How it was built

Starting with GSQ-RCO IQ2_S-MTP, I replaced 131 matrices with weights read directly from OrcaRouter's BF16 SafeTensors:

• 1 input embedding matrix;
• 64 mlp.down_proj, 48 linear_attn.out_proj, and 16 self_attn.o_proj matrices;
• 2 MTP attention and MLP output projection matrices.

The other 735 of 866 tensors, runtime metadata, and tokenizer were preserved byte for byte from GSQ. The hybrid uses selective tensor replacement, without averaging weights.

The replacement matrices were quantized from BF16 with llama.cpp's ggml kernels, using GSQ's type allocation map and official imatrix (activation statistics from the base Qwen model). The 48 linear attention output projections also received the converter's required column reordering.

This is conventional quantization with GSQ's published map and calibration; the full GSQ-RCO refinement and allocation optimization apply to the original GSQ process. The resulting file retains the original GSQ IQ2_S-MTP size.

Build checks covered tensor counts, types, shapes, metadata, column reordering, finite BF16 source values, and byte-for-byte preservation of the 735 retained tensors. The model also passed loading and generation checks.

Quality tests

Short benchmark screening

I used the first 30 items from each BenchKit suite, in dataset order, with one generation per item and matching settings for both models.

HumanEval's raw scores were 25/30 and 21/30, respectively. BenchKit's extractor changed indentation in some responses; applying the same documented indentation normalization to both models brought each to 30/30, using the existing generations and unchanged program logic. Tests ran in an isolated, disposable WSL environment.

Settings: BenchKit 037ec91, temperature 0, medium reasoning, reasoning budget 1,024, total generation limit 4,096, and ngram + MTP. There were no empty final responses or timeouts; one IFEval response reached the total limit and passed.

These small subsets provide a local comparison. Their item selection and BenchKit evaluators—including its own IFEval checkers—differ from full published benchmark runs.

Reasoning budget

The hybrid's two IFEval failures on items that GSQ passed involved deliberation text appearing in the final answer and breaking formatting constraints. Repeating those two items on both models with a 2,048 reasoning budget resolved both failures; GSQ continued to pass.

Based on that small diagnostic, 2,048 is a useful starting point for strict-format tasks, with room left for the final answer. The table retains the original scores at 1,024.

A real coding task

Both this hybrid and my earlier GSQ + Huihui UD-IQ2_S version continued the same website-development conversation, with 30.6K tokens of initial history, identical site copies, and two new requests. This hybrid produced both final responses and passed 15/15 functional checks after the second turn. The earlier version passed 2/15 and used the six-generation limit per turn on tool calls without a final response. This comparison covers one task within that execution budget.

Performance — RTX 3060 12 GB

Rates use total tokens divided by total corresponding time from llama-server logs. Decode includes reasoning and tool calls where present; prefill measures prompt processing. Context grows during generation.

The short-prompt figures cover 90 requests per model, using the screening settings above and excluding the later reasoning diagnostic. The real conversation generated 9,564 tokens, with full GPU offload, prefix reuse, ngram-mod + MTP, and no detected concurrent workload. Peak VRAM was 11,536 MiB, with 578 MiB minimum free.

These are initial usage references. Context length and sampling differ between scenarios, and the small speed difference between models in screening is within the uncertainty of this unrepeated run. The long-context measurement starts at 30.6K tokens; performance with 65K occupied remains to be measured. Required VRAM also includes KV cache, buffers, and speculation.

Settings used for the 30K-history measurement

llama.cpp b10715, commit 92cedc8679d145902ead3f006258e8672eac11e6, from the local Unsloth installation.

--ctx-size 66560
--batch-size 2048
--ubatch-size 128
--gpu-layers 66
--fit off
--parallel 1
--flash-attn on
--cache-type-k q4_0
--cache-type-v q4_0
--no-context-shift
--jinja
--reasoning on
--reasoning-format deepseek
--reasoning-effort medium
--reasoning-budget 1024
--spec-type ngram-mod,draft-mtp
--spec-draft-n-max 3
--spec-draft-p-min 0.0
--spec-ngram-mod-n-match 24
--spec-ngram-mod-n-min 48
--spec-ngram-mod-n-max 64

Sampling: temperature 0.85, top-p 0.95, top-k 20, min-p 0, repetition penalty 1.05, presence penalty 0. Context caching used the runtime default (enabled); no vision projector was loaded. These measurements used budget 1,024; the 2,048 suggestion comes from the separate quality diagnostic.

Evaluation scope

Testing focused on text, reasoning, and the coding task above. Abliterated behavior was retained in manual use; systematic refusal, language-specific, general tool-calling, and multimodal evaluations remain open. Aggressive quantization can affect accuracy, so the model is best treated as an experimental local-use option.

Credits and license

This independent derivative builds on the original teams' training, abliteration, and quantization work:

• Qwen/Qwen3.8-27B: original model.
• OrcaRouter Uncensored: abliterated BF16 weights, pinned source revision.
• DASLab GSQ-RCO GGUF: base GGUF, allocation map, and imatrix, pinned source revision.
• llama.cpp and Unsloth: quantization and inference tools and local environment.
• BenchKit: screening runner, pinned revision.

License: Apache-2.0, as declared by the weight sources, with attribution retained.

Usage reports are welcome—please include your GPU, runtime version, actual context usage, speculation settings, and a reproducible example.