Qwen3.8-27B redashes SSMFIX · INT8 W8A16 · BF16 MTP

A conv1d-repaired, Ampere-optimized quantization for dual RTX 3090 inference.

Base model · Qwen3.8-27B · vLLM · llm-compressor

[!NOTE] This is a numerical W8A16 quantization of redashes/Qwen3.8-27B-BF16-SSMFIX, a conv1d-repaired version of Qwen/Qwen3.8-27B. All model credit belongs to redashes (the SSM repair) and Qwen (the base model); refer to the upstream model card for architecture, capabilities, and usage guidance. This repository changes numerics only.

[!TIP] This build targets vLLM on Ampere-class GPUs where native FP8 tensor-core execution is unavailable. It retains the model's vision tower, thinking controls, native context, and the byte-preserved BF16 MTP head. The SSM scale-drift repair lives in linear_attn.conv1d.weight tensors, which are convolution weights outside the W8A16 target set — so the repair is carried through at source precision.

Relationship to the sibling quantization

This checkpoint is the conv1d-repaired counterpart of lued/Qwen3.8-27B-INT8-W8A16-MTP. The repositories share:

• the identical W8A16 compressed-tensors recipe (400 packed GEMMs, same BF16 preservation set, group-128 symmetric data-free RTN);
• the identical native BF16 MTP head (15 tensors, byte-identical to Qwen's original MTP — same sha256 as the sibling release);
• the identical tokenizer, chat template, processor configs, and bounded-shard layout;
• the same total checkpoint footprint (29.44 GiB).

The only deviation is the base language-model weights: this checkpoint's LM weights come from redashes's conv1d-repaired BF16 model instead of stock Qwen3.8-27B.

SSM scale-drift repair (inherited from the base)

Per the upstream card, the source applies per-layer α-scaling to the anomalous linear_attn.conv1d.weight tensors in Qwen3.8-27B, following the methodology first disclosed by LuffyTheFox (Sig-ScaleSync) and independently re-implemented by redashes: 8 layers (52/53/56/57/58/60/61/62) rescaled with factors 0.481–0.653. Upstream-controlled evaluation showed the patch moving generative metrics (TruthfulQA-gen +6~8pp, MT-Bench +0.42) while slightly reducing closed-book knowledge (CMMLU −1.8pp). These are the upstream card's claims, not measurements made here. The W8A16 step touches only stored numerics of the 400 GEMMs; the conv1d convolution weights are preserved at BF16, so the repair is bit-exact in this release.

Quantization fidelity

Measured on this exact checkpoint against its own BF16 teacher (the redashes source) on 2026-08-18 with tools/measure_kld.py (teacher-forced, full-vocabulary FP32 log-softmax, FP64 accumulation, nats/token; PyTorch 2.11.0+cu128, Transformers 5.10.1, compressed-tensors 0.17.1):

The KLD result measures checkpoint-weight drift over 467 teacher-forced positions (fixed 8-prompt short suite). It is not a functional quality score; tool use, coding, multimodal quality, and long-context recall require separate behavioral evaluation.

Short-prompt suite

kld_result.json in this repository is the single source of truth for these numbers.

Checkpoint profile

MTP preservation

The MTP head is the exact BF16 MTP from the original Qwen3.8-27B — never touched by the SSM repair and never re-quantized:

• 15 tensors in model_mtp.safetensors (849,400,392 bytes).
• sha256 1d8268aa85ace093a561e3e7b63b9d390dac1cd55a90cd55b5ec509c3c9da9fe — identical to the MTP bytes in the Qwen3.8-27B source, in the AEON source, and in the sibling release lued/Qwen3.8-27B-INT8-W8A16-MTP.
• All 15 tensors verified bfloat16, shape-equal, and bit-equal to the redashes source (strict audit, not a count check).

Reproduction contract

The immutable source is redashes/Qwen3.8-27B-BF16-SSMFIX commit 859730a4ae4d6ed793d0f427aa95d49135334bda. The exact selective W8A16 configuration is recorded in recipe.yaml and config.json. A conforming rebuild must take native MTP from that source, reproduce the source-derived packed projection set, and prove all 15 MTP tensors bit-identical before resharing for publication.

Status

Built on 2026-08-18: both release audits PASS, KLD measured, published to lued/Qwen3.8-27B-redashes-SSMFix-INT8-W8A16-MTP at commit e4ff78c49d2717023aa9965fd1aa85be2387407d, and registered in the HF cache (link_hf_cache.py sync PASS). Serving validated the same day: boots through llama-swap from the lued/* cache entry, loads the CompressedTensorsWNA16 → MarlinLinearKernel path, detects the native MTP draft model (method=mtp, 3 speculative tokens), and serves a smoke request. The BF16 parent source remains cached; deletion is the pipeline's final step and has not been requested.