Qwen3.8-27B-BF16-SSMFIX-apostate-GSQ-RCO-IQ3_S

An optimized, uncensored 27B multimodal GGUF binary for llama.cpp. Built from redashes/Qwen3.8-27B-BF16-SSMFIX-apostate using a per-tensor mixed-precision search (GSQ + RCO), this quantization delivers full 27B-class reasoning and vision capabilities within a 12.0 GB weight budget, comfortably fitting 16GB and 24GB GPUs.

---

What Is Unique About This Binary?

Unlike generic homogeneous quantization (e.g., standard Q4_K_M or Q3_K_M), this model is provided as a single, highly tuned IQ3_S mixed-precision binary engineered to maximize perplexity retention at an average density of ~3.52 bits per weight:

Inference Performance

Benchmarked on a single NVIDIA RTX 3080 20GB (llama-kvmem-server, kv q8_0, speculative draft-mtp n=3):

• Generation / Decode: ~48.5 t/s
• Prompt Processing / Prefill: ~1150 t/s

---

Quick Start & Usage (llama.cpp)

This repository contains both the base language model with MTP head and the matching multimodal projector:

1. Qwen3.8-27B-GSQstyle-IQ3S-mtp.gguf — Language model and MTP head.
2. mmproj-Qwen3.8-27B-Q5_K-MIX.gguf — Multimodal vision projector.

1. Interactive Text Chat (with MTP Speculative Decoding)

llama-cli \
  -m Qwen3.8-27B-GSQstyle-IQ3S-mtp.gguf \
  --spec-type draft-mtp --spec-draft-n-max 3 \
  -ngl 99 \
  -c 16384 \
  --temp 0.7 \
  -p "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nWrite a technical breakdown of zero-knowledge proofs.<|im_end|>\n<|im_start|>assistant\n"

Thinking Tokens: By default, Qwen3.8 / Qwen3.5 architecture produces <think>...</think> internal monologue before the final answer. If you want direct answers without thinking, suppress the reasoning prompt in your system template or instruct the model directly.

2. Vision / Multimodal Inference

To process images, pair the language model with the included mmproj file:

llama-mtmd-cli \
  -m Qwen3.8-27B-GSQstyle-IQ3S-mtp.gguf \
  --mmproj mmproj-Qwen3.8-27B-Q5_K-MIX.gguf \
  --image /path/to/diagram.png \
  -ngl 99 \
  -p "Describe the architecture shown in this diagram in detail."

3. Running as an OpenAI-Compatible API Server

llama-server \
  -m Qwen3.8-27B-GSQstyle-IQ3S-mtp.gguf \
  --mmproj mmproj-Qwen3.8-27B-Q5_K-MIX.gguf \
  --spec-type draft-mtp --spec-draft-n-max 3 \
  -ngl 99 \
  -c 32768 \
  --port 8080 \
  --host 0.0.0.0

---

Lineage & The SSMFIX Foundation

This quant is converted from redashes/Qwen3.8-27B-BF16-SSMFIX-apostate, which in turn inherits from redashes/Qwen3.8-27B-BF16-SSMFIX.

Why SSMFIX Matters

Qwen3.8-27B is a hybrid linear-attention model consisting of:

• 64 layers: 16 full-attention layers and 48 linear-attention/conv1d state-space layers.
• Stock Qwen3.8 checkpoints exhibited numerical instability and output degradation across 8 anomalous SSM layers.
• SSMFIX-v2 applied targeted per-layer $\alpha$-scaling and repaired conv1d weights, stabilizing the state space blocks before any unlearning or quantization took place.

---

Uncensored & Abliterated: What is Apostate?

This model is effectively uncensored for creative, technical, red-teaming, and unrestricted question-answering workflows. In human testing, refusal patterns ("I cannot fulfill this request...") are absent across general instruction tasks.

The unlearning was performed using the Apostate framework by heterodoxin, specifically its KCRN (Kernel Constraint Rank-Nullity) projection method.

Why Apostate KCRN Beats Heretic

Most common abliteration pipelines (such as Heretic) suffer from serious design limitations:

1. Heretic's Reasoning Blind Spot: Heretic tunes against a first-token KL divergence objective. On reasoning models (like Qwen3 and Qwen3.8), the first token emitted is almost always <think>. As a result, first-token loss is completely blind to whether the actual chain-of-thought or subsequent output has been damaged, reporting near-zero divergence while silently corrupting model weights.
2. Full-Position Reloaded Validation: Apostate computes KL divergence in float32 across every non-padding token position, comparing the base model against the reloaded, baked checkpoint on an independent benign dataset.
3. Analytic Null-Space Projection ($\Delta W \perp Q_b$): Rather than blindly subtracting refusal vectors globally (which introduces 0.1+ KL damage across general reasoning tasks), KCRN:
   • Identifies a low-rank refusal subspace $Q_b$ across 42 critical weight projections (35 mlp.down_proj, 5 linear_attn.out_proj, 2 self_attn.o_proj).
   • Analytically projects weight modifications onto the null-space of $Q_b$ in a single closed-form solve.
   • Preserves benign generation subspaces mathematically, bounding KL divergence by design.

Verified Divergence & Refusal Rates

• Refusal Rate: Dropped to 7% (93% delivery rate) on standard harmful evaluation suites (harmful_1000[800:900]), down from ~99% on the aligned base. The remaining refusals are concentrated solely in strict legal/crisis boundaries.
• True KL Divergence: Independently verified at 0.00598 calibration / 0.00392 heldout nats/token against the SSMFIX base (well below the 0.05 degradation threshold, confirming the preservation of core capabilities).

---

Capability Retention Benchmarks

The underlying BF16 unlearned checkpoint maintains virtually all of the official Qwen3.8-27B reasoning, mathematics, and bilingual capability:

---

Repository Files

---

Disclaimer & Legal Notice

This is an experimental, uncensored research artifact created via weight unlearning and mixed-precision quantization. The model will answer prompts without safety alignment filters. Users are solely responsible for ensuring deployment and generations comply with all local laws and the base model's Apache-2.0 license terms.