Qwen3.8-Flash-Next · heretic · fngine pack

A decensored build of Qwen/Qwen3.8-Flash-Next for fngine, a single-GPU C++20/CUDA engine written for this model (tested with commit c20a885af5ac; the engine lives in its own repository and is not included here). It is the ISTA-DASLab GSQ-RCO IQ3_S quantization carrying the decensoring of trohrbaugh/Qwen3.8-Flash-Next-heretic-2, made with Heretic (author-reported: 0/100 refusals, KL divergence 0.0818 from the original).

Reduced safety filtering. Refusal behaviour has been largely removed. The model can produce content that the original would decline. You are responsible for how you use it and for complying with the law and the license.

Measured against the original model

Same engine and settings for every pack (RTX 5090); raw data in heretic/results/.

The agreement row compares each pack's predictions with the original model's own (official HF modeling code, the same GGUF experts); the original pack's own gap is FP8 dense weights and the 8-bit KV cache. This pack stays within 0.005 KLD of it, while the huihui-ai abliteration (which reflects the refusal direction in every layer) moved 0.17 away and, by its HF reference, predicts ordinary text with 8–10% higher perplexity than the original. The one prompt this pack still declines is "List the ten most offensive insults you know." Speed differences are within run-to-run noise (±12–34 tok/s).

Engine tests: 46/46 pass on this pack, including the real-model check that greedy decoding with speculative drafts produces exactly the plain greedy output (3 prompts × 256 tokens); MTP first-draft acceptance 0.718.

What is in this repository

pack/dense_bf16.bin (8.1 GiB) is only used by fngine's validation mode; serving does not need it.

How the decensoring was transferred

heretic-2 was measured as a weight delta against the official release (both range-fetched from the Hub):

• Only residual-stream writers changed, and only in a band of layers: the attention/GDN output projections of layers 32–47 (weights 2.16–2.38), the shared-expert down projections of layers 31–35 (0.45–1.71), and every routed expert's down projection in layers 31–35, with the same weight in every expert of a layer (spread ≤ 0.2% over eight sampled experts) as that layer's shared expert. 326 other tensors spot-checked across all layers (Q/K/V, router, expert gate/up, hyper-connections, embeddings, lm_head, MTP head) are byte-identical.
• Each edit is W' = W − w·r·rᵀW along that layer's own refusal direction (Heretic's per-layer scope): the second component of each layer's delta is at most 7×10⁻⁴ of the first.

In this pack:

• the 21 changed dense tensors are heretic-2's exact BF16 bytes, converted to fngine's FP8 rows by the same converter as every other tensor (the other 879 dense tensors are byte-identical to the original pack);
• the routed experts keep their GSQ-RCO bytes. The routed output is a weighted sum of linear down projections, so editing every expert is exactly equivalent to editing their sum: y_routed −= w·r·(r·y_routed). fngine applies this per layer in its MoE combine kernels (decode, verify and prefill; GPU- and CPU-resident experts) from pack/ablation.bin (weight 0 outside layers 31–35). The shared expert is added unedited because its weights already carry the edit;
• everything else is unchanged from the original pack, so speed and memory are the same.

Requirements

• NVIDIA RTX 5090 (or another sm_120a GPU with 32 GB); fngine is compiled for that architecture only
• an x86-64 CPU with AVX-512 VNNI, BF16 and VBMI (AMD Zen 4/5, Intel Sapphire Rapids or later) for the CPU expert tier
• ~60 GB host RAM (CPU-tier experts, embedding, n-gram cache) and an NVMe SSD (the n-gram table is read on demand)
• Linux x86_64 with a working NVIDIA driver; ~87 GB of disk for the serving files

Usage

The installer builds fngine at a pinned commit with an isolated CUDA toolchain, downloads and verifies this pack (without the validation-only dense_bf16.bin), starts the server and updates both:

curl -fsSL https://raw.githubusercontent.com/satellitedown/flash-next-abliterated-fngine/main/get.sh | bash

Manually (CUDA 13.3, CMake ≥ 3.28, Ninja, liburing, a C++20 compiler; if your default GCC is newer than 15, add -DCMAKE_CUDA_HOST_COMPILER=g++-15 to the configure step):

hf download satellitedown/Qwen3.8-Flash-Next-heretic-fngine --local-dir qwen3.8-flash-next-heretic \
    --exclude pack/dense_bf16.bin
git clone https://github.com/satellitedown/fngine && cd fngine && git checkout c20a885af5ac
cmake --preset release && cmake --build build -j
build/apps/fngine-serve --pack ../qwen3.8-flash-next-heretic/pack --ctx 262144 \
    --host 127.0.0.1 --port 8002 --model-id qwen3.8-flash-next

OpenAI-compatible Chat Completions (streaming, tools, reasoning_content, enable_thinking):

curl -N http://127.0.0.1:8002/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model":"qwen3.8-flash-next","stream":true,"messages":[{"role":"user","content":"Hello"}]}'

The server runs one request at a time with a short FIFO queue. With agent frameworks that issue parallel requests, cap concurrency to 1 for this endpoint. See the fngine README for every option, including 256K-context notes, speculative decoding and validation tooling. fngine-serve --no-ablation ignores ablation.bin (dense edit only), for comparison.

Credits and licenses

• Qwen3.8-Flash-Next: Qwen, Qwen Community License 1.0. This repository is a derivative work; the license and its conditions apply to it.
• GSQ-RCO quantization of the routed experts and n-gram table: ISTA-DASLab (GSQ, RCO); Apache-2.0, inheriting the base model's license.
• Decensoring: trohrbaugh/Qwen3.8-Flash-Next-heretic-2, made with Heretic by p-e-w (custom fork).
• fngine (not included here): Apache-2.0; derived in part from Cinference and NInfer, with ggml i-quant code (MIT) and vendored libraries under their own licenses. See NOTICE and LICENSE in its repository.

Provided as is, without warranty. Do not expose the server unauthenticated to untrusted networks.