Qwen3.8-27B Huihui W4A16 AWQ — 19.55 GB / 18.21 GiB

This is an activation-aware AWQ checkpoint of huihui-ai/Huihui-Qwen3.8-27B-abliterated. The current main branch follows upstream revision 739e3c5b89849f6c238ce1e5b70008612ae42cdd.

The language-model GEMMs are packed as asymmetric W4A16 with group size 128. Vision, native MTP, the LM head, and linear-attention in_proj_a and in_proj_b remain in source precision.

[!IMPORTANT] The current revision passed complete artifact audit and native GPTQModel semantic generation on an RTX 3090. It has not yet replaced the live model for an exact 4x V100 runtime smoke. The older V100-validated checkpoint is permanently available at v1-d42ca897.

This is a community quantization, not an official Qwen or Huihui release.

Versions

What changed in the mother checkpoint

Compared with the mother revision used for v1, upstream narrowed the abliteration scope. A tensor-level comparison found 28 matrices restored to official Qwen values in layer indices 15, 16, and 52–63: 14 MLP down_proj, 10 linear-attention out_proj, and 4 self-attention o_proj matrices. The remaining 70 Huihui-modified matrices are in indices 17–51. Vision, MTP, configuration, tokenizer, template, and weight index were unchanged.

Frozen source and quantizer

The pinned mother contains 18 weight shards, 1,199 indexed tensors, and 55,562,855,904 indexed tensor bytes. SOURCE-MANIFEST-BF16-MOTHER.json records the pinned source files and hashes.

Quantization recipe

The recipe uses joint in_proj_qkv/in_proj_z AWQ smoothing plus zero-centered RMSNorm-aware BF16 compensation for the retained split-GDN in_proj_a/in_proj_b projections.

Validation of the current version

The v2 release passed these gates:

• 400/400 target modules packed;
• 400 qweight, 400 qzeros, and 400 scales tensors stored as INT32, INT32, and BF16;
• 799 retained source tensors present;
• 350 structurally protected tensors verified byte-for-byte;
• 96 retained split-GDN BF16 tensors compensated and audited;
• all targets used 202,750 calibration token samples;
• native GPTQModel 7.3.2 BACKEND.TORCH FP16 load and semantic smoke passed on an RTX 3090, including exact NATIVE_AWQ_OK output and a natural Chinese response;
• every release file is covered by SHA256SUMS.

The current v2 weights do not yet claim exact 1Cat-vLLM/SM70/V100 runtime validation. The v1 tag retains the earlier V100-validated artifact and its historical results.

Download

Current version:

hf download leoncca/Qwen3.8-27B-Huihui-AWQ \
  --revision v2-739e3c5b \
  --local-dir Qwen3.8-27B-Huihui-AWQ-v2

Previous V100-validated version:

hf download leoncca/Qwen3.8-27B-Huihui-AWQ \
  --revision v1-d42ca897 \
  --local-dir Qwen3.8-27B-Huihui-AWQ-v1

Verify either download from inside its model directory:

sha256sum -c SHA256SUMS

Runtime flags depend on the inference engine. Do not assume that a loader which supports older dense AWQ models already supports Qwen3.8 multimodal, linear-attention, native-MTP, and SM70 execution.

License and use

The included model files are released under Apache-2.0, following the source repository. Review the upstream model card and license before use.

The Huihui mother checkpoint intentionally changes refusal behavior. This checkpoint is not safety-tuned, and its outputs may be inaccurate, offensive, or unsafe. Downstream users are responsible for evaluation, safeguards, and legal compliance in their own deployment context.

Acknowledgements

• Qwen
• huihui-ai
• GPTQModel
• 1Cat-vLLM