Huihui Qwen3.8-27B Abliterated · Ternary-Bonsai-2 — NInfer

Format repackaging of huihui-ai's abliterated Ternary-Bonsai GGUF into a single-file .ninfer container for the NInfer engine.

The abliteration is not ours. huihui-ai ablated layers 22–52 (0-based) of the PrismML Ternary-Bonsai-2-27B weights; this repository only repackages them. See Attribution and the usage warnings carried over below.

Why this tier needed re-quantization

huihui-ai's published Ternary-Bonsai-PQ2_0.gguf has 54 tensors that lost their ternary encoding (ffn_down.weight ×31 and ssm_out.weight ×23 became Q3_K/Q2_K) — they had to de-quantize those tensors to ablate them. A mixed ternary/k-quant file cannot enter the .ninfer container, whose runtime only reads t2_g128_fp16.

This artifact was therefore built from huihui-ai's f16 release (which retains the full ternary layout and the complete PrismML Hadamard metadata) and re-quantized to PQ2_0 with PrismML's own quantizer:

llama-quantize --tensor-type-file ssm_bf16.txt \
               --token-embedding-type PQ2_0 --output-tensor-type PQ2_0 \
               huihui-bonsai-f16.gguf out.gguf PQ2_0

Result: PQ2_0 ×402 + F32 ×353 + BF16 ×96 — the same type breakdown as PrismML's shipped PQ2_0 file. The ssm_alpha/ssm_beta controls stay BF16 because the runtime requires full precision there.

No PTQ1_0 artifact is published here

The v3 container has a single ternary format, t2_g128_fp16, fixed at 2 bits per weight. It accepts both source packings — the converter's read_ternary decodes decode_pq2_0 and decode_ptq1_0 natively — but both decode to the same trits, which are then stored in that one 2-bit layout. The density PTQ1_0 saves upstream (1.75 vs 2.13 bits/weight) is undone the moment the rows are read.

An artifact converted from the PTQ1_0 source is therefore byte-for-byte identical to one converted from PQ2_0: the same 792 objects, the same payload sha256. Publishing both would cost 7.29 GiB for a duplicate, so only PQ2_0 is kept.

(This was not true of the earlier v2 container. The ninfer-ada-ternary lineage kept PQ2_0_G128 and PTQ1_0_G128 as separate storage formats, so a PTQ1_0 artifact was about 1.17 GiB smaller. v3 has no such distinction.)

Files

A text-only and an MTP-bearing variant of the same ternary model.

SHA256SUMS carries the full digests.

One complete container: text tower (64 layers, 4:1 linear/full attention mix), proposal head, tokenizer, chat template and generation config. The two variants differ only in the MTP head, which the -mtp file carries and the other does not. Neither variant has a vision tower; the Ternary-Bonsai source has none.

Where the MTP head comes from

The -mtp files embed Qwen3.8-27B's multi-token-prediction head, which the engine enables with --spec mtp. The head is neither trained by us nor re-quantized: it is copied byte for byte from the v2 container in Barding-Defense/Qwen3.8-27B-huihui-abliterated-groupwise-int-NInfer, stored as q8_g32_fp16, the format the bonsai2_27b_ternary recipe assigns to MTP. It comes from the same huihui abliterated base as the text tower.

The graft rewrites only the container directory. The text tower's 7,381,449,216 payload bytes are carried across untouched, with an identical sha256 before and after. Twelve tensor objects are added, 451,267,584 bytes in total.

Engine

Needs a NInfer engine build that reads the ternary t2_g128_fp16 representation with the PrismML Hadamard (normalized Sylvester–Walsh) rotation. This is a different dialect from the gguf_blocks_v1 artifacts — the two are not interchangeable.

Converted with the bonsai2_27b_ternary recipe from Ryan-gsq/ninfer-16g-5070ti-5080-5090-qwen3.8-27b-gsq-rco (Apache-2.0, forked from iamwavecut/ninfer-all).

Attribution

Qwen3.8-27B (base)                    Qwen/Qwen3.8-27B                     Apache-2.0
Ternary-Bonsai-2-27B (ternary)        prism-ml/Ternary-Bonsai-2-27B-gguf   (see its LICENSE)
Abliteration (layers 22–52)           huihui-ai/Huihui-Qwen3.8-…-GGUF      Apache-2.0
  re-quantized to PQ2_0 from its f16 release
MTP head (-mtp files only)            Barding-Defense/…-groupwise-int-NInfer   Apache-2.0
This repo                             .ninfer repackaging

See NOTICE for the full chain and the exact conversion steps.

Usage warnings (carried over from the abliterated source)

The source model's safety filtering has been significantly reduced. Outputs may be sensitive, controversial or inappropriate. Not suitable for public-facing settings, for use by minors, or for applications requiring high safety. Users are solely responsible for compliance with applicable law and for reviewing generated output. Recommended for research, testing and controlled environments only.

Intended use

Research and local inference. Not validated for production or safety-critical use.

中文说明