Qwen3.8 Flash Next Uncensored AWQ g32 + QSA FP8 E4M3 KV

2026-10-16 update: 137.07 GB (127.66 GiB) across 86 model shards.

AWQ-format derivative of orcarouter/Qwen3.8-Flash-Next-Uncensored. All routed-expert projections have been requantized from the same BF16 mother with damped GPTQ, using real captured activations. The previous release's s1 smoothing and all non-routed compensation carriers are preserved byte-for-byte. PLE uses the fixed official FP8 weights. Existing QSA FP8 E4M3 K/V scales are preserved; they were not recalibrated for these new expert weights.

On a fixed 66-record text subset, mean top-20-plus-tail KL to the BF16 reference decreases from 0.0303 to 0.0214 (about 30%), and top-1 agreement rises from 96.34% to 97.08%. These are bounded fidelity checks, with the approximation and runtime limitations below; they are not general capability benchmark scores.

Recommended runtime on V100

On NVIDIA V100 (SM70) we serve this checkpoint with ReVolta, our open-source vLLM-based engine for 4 × V100 32 GB. It contains the SM70 support this checkpoint needs (AWQ g32 MoE kernels, QSA with calibrated FP8 E4M3 KV, the FP8 MTP draft and multimodal input), and a pre-release build of it ran the validation below. Its README has the launch command and the supported settings. On Ampere or newer GPUs, see Stock vLLM under Loading requirements.

Checkpoint

• Mother revision: 8336e613ea508b13c2159bd0f68965d97a606b95
• Official FP8 PLE revision: bcd9f01ddc9cff2316eb84281bebcd5b058bddce
• Format: standard asymmetric AWQ W4A16, group size 32, integer zero point, GEMM packing
• Quantized: gate_proj, up_proj, down_proj of all 512 routed experts in all 48 layers (24,576 experts)
• Optimizer: damped GPTQ, gate/up damping 0.3 and down damping 1.0
• Expert scales: FP16, changed from the previous release's BF16 scales
• s1 and all 384 non-routed compensation carriers: byte-identical to the previous release, including original BF16 storage
• Indexed tensors: 222,773, including 24 target and two default-BF16-MTP FP32 scalar K/V scales
• Shards: 48 expert layer files, three retained base files, 33 PLE files and two K/V-scale files
• Merged index SHA-256: c4e151b8bd9906dae382e68b589c98f67f58f5bcc2e6aaaaa6d52dea04dffd78
• Target K/V-scale shard SHA-256: a709ed27825e1b5e97e8b2127671e0d0b7715f014c5db71cc414a3b96af76be1

Router, shared expert (gate/up/down), shared-expert gate, HC norm, and both HC down-projection carriers retain the previous release's exact bytes. They have not been converted to FP16 storage. Other non-routed tensors are retained from the source checkpoint; no new PLE conversion is performed. The root index also includes the source checkpoint's two default BF16 MTP K/V scales, which were absent from the previous root HF index. This is separate from mtp-fp8/.

Identical index hashes can describe the same tensor names and shard layout in different checkpoints; they do not prove identical weights. Use SHA256SUMS for complete file identity. Public provenance and results are in MODEL_PROVENANCE.json, VALIDATION.json and EXPERT_COVERAGE.json.

Why AWQ g32

W4A16 provides a weight-only INT4 path with 16-bit activation/compute precision. The native g32 layout matches width-640 routed experts under TP4: 640 / 4 = 160, which is divisible by 32 but not 128.

AWQ describes the serialized layout here; the new expert weights were optimized with damped GPTQ. They do not require a GPTQ checkpoint loader. Bits, group size, asymmetric zero and GEMM packing are unchanged. Loaders must accept FP16 expert scales and preserve the mixed source dtypes of non-routed parameters.

Calibration

Weight calibration uses real activations captured from the previous release served in vLLM, on the public80-v1mix20 calibration corpus, across all 48 layers (169,614 calibration rows per layer). Previous s1 is recovered from the existing carriers by joint least squares and checked against the saved s1. Per-expert s2 is frozen from the W2.5 calibration search; this requantization does not search a new s1/s2 or tune on heldout results.

The Hessian and damped GPTQ use the captured previous-checkpoint routing. Scales are rounded to FP16 before code selection, integer zero points are stored, and codes follow clip(round(w / scale + zero), 0, 15) on the current GPTQ residual with feedback. Standard AWQ GEMM packing is used for all experts.

All experts are quantized, with no routing bypass. Natural sample counts are published by layer and expert in EXPERT_COVERAGE.json. Only the seven zero-natural-coverage layer/expert pairs use 2,048 equally spaced rows from their layer as a proxy. Every expert with natural coverage uses its natural samples plus damping; a low positive count does not trigger proxy replacement. Unchanged router weights do not imply identical end-to-end routing after earlier expert outputs change.

The preserved target K/V scales were previously calibrated using 14 input shards, 4,904 prompts and 4,888,771 processed prompt tokens. The 24 finite, positive FP32 scalars cover K/V at all 12 QSA layers, ranging from 0.0170375295 to 0.0819614977. They are KV-storage scales, separate from AWQ weight scales and PLE scales. Reusing them does not mean new scale calibration.

Loading requirements

The loader must support:

• Qwen4Exp per-expert asymmetric AWQ W4A16 g32 in GEMM layout, including FP16 expert scales;
• the official FP8 PLE tensors and the complete merged tensor index;
• the 24 target QSA K/V scale entries through model.safetensors.index.json, plus the two draft entries if using default BF16 MTP;
• full Qwen4Exp multimodal handling, including MRoPE and video-token-pruning initialization, for images and videos.

Tested on four 32 GB NVIDIA V100 GPUs (TP4, FP16 compute, FP8 E4M3 KV storage) with a pre-release build of ReVolta. On V100, E4M3 refers to KV storage, not native FP8 attention compute. Request fp8_e4m3 explicitly in a compatible runtime; downloading the scales alone does not enable FP8 KV. Generic AWQ support alone is insufficient for this architecture. No stock-runtime compatibility or default launch configuration is implied.

Stock vLLM. Upstream vLLM (checked in v0.31.0 and v0.32.0) runs the Qwen4Exp QSA layers only in BF16, which needs compute capability 8.0 or newer. config.json therefore declares bfloat16, so the default --dtype auto selects BF16 on Ampere or newer GPUs and no extra flag is needed. A user has reported that this checkpoint runs on stock vLLM v0.31.0 with BF16 compute; we have not run it ourselves. If you see NotImplementedError: Qwen4Exp QSA currently requires BF16, the run is using FP16, for example from an older copy of config.json or an explicit --dtype float16. Stock vLLM cannot run this model on pre-Ampere GPUs such as V100 or T4; there, vLLM falls back to FP16 automatically because the GPU has no BF16. In BF16 mode the FP16 expert scales are rounded to BF16 on load (at most 0.4% relative per scale); the GPU fidelity figures below were measured with FP16 compute, not BF16.

The bundled config keeps native 262,144-token context and default RoPE. The BF16 comparison below used matched YaRN factor 2 runtime settings; it does not change the bundled defaults or establish a 512K capability score.

Validation

BF16 fidelity comparison — 2026-10-16

The same stratified 66-record subset provides 39,101 scored tokens. BF16 reference distributions come from a transformers CPU full-vocabulary forward over the original text model, expanding BF16 weights exactly to FP32 for the main computation. The quantized arms were served in the deployment configuration listed under Loading requirements (TP4, FP16 compute, E4M3 KV). The comparison therefore includes runtime arithmetic and KV effects as well as weight quantization.

KL is KL(quantized arm || BF16), in nats per scored token. Means are weighted by valid token count. Top-1 agreement compares captured rank 1 with the BF16 FP32 argmax before logprob storage rounding.

Instructions are English, Chinese and code instructions. Dialogue also includes math reasoning; Instructions is a subset of Dialogue. All = Dialogue + Documents. The overall coarse-grained KL decrease is 29.52% (rounded to 30%).

Approximation limit: only the quantized arm's top 20 probabilities and total tail mass are captured. Merging the tail into one bin gives an exact coarse-grained KL and a lower bound before numerical uncertainty. It is also a point approximation to full KL if the unknown arm tail follows the BF16 conditional tail distribution. Tail differences are not measured, so this does not establish a 30% reduction in full-vocabulary KL. Accounting for FP16 reference storage rounding and an explicit serving-logprob tolerance of ±1e-6, conservative full forward-KL bounds for All are [0.030037, 0.332035] previous and [0.021093, 0.318083] new. These are numerical/unknown-tail bounds, not statistical confidence intervals. Exact group results are in VALIDATION.json.

Exact full-vocabulary check (weights only): the same 66 records were also run on CPU in FP32 with this checkpoint's AWQ expert weights dequantized exactly (w = (q - z) * s) and every other tensor read from the checkpoint, so complete distributions are available. Exact full-vocabulary KL(quantized || BF16) is 0.0232 for this release and 0.0323 for the previous one (-28%); KL(BF16 || quantized) is 0.0228 and 0.0314. On these CPU distributions the top-20-plus-tail method reads 7-10% low. This measures weight quantization alone; it excludes GPU kernel, tensor-parallel and KV-cache effects.

This is a fixed stratified subset, not the full corpus or a broad benchmark. The reference is text-only; these KL results do not evaluate vision or MTP. CPU FP32 reference arithmetic differs from GPU FP16 serving; it cannot isolate quantization error from kernel, tensor-parallel reduction or KV effects.

NLL lesson: on the external instruction texts, the previous release's NLL was 1.451809, lower than the original BF16 model's 1.471955. The new release's NLL is 1.475266 while its distribution is closer by the reported KL and top-1 checks. NLL relative to the previous quantization is therefore not a quality ranking or a BF16-fidelity measure.

Long-context check — 2026-10-17

Three natural documents were scored against a BF16 CPU reference: an English novel to 32K tokens, a Chinese novel to 64K and the SQLite amalgamation to 128K (every 16th position, 14,336 positions in total; the quantized side was served in the configuration above, and the BF16 reference uses the same YaRN factor 2). Top-20-plus-tail KL(quantized || BF16) is 0.0238 with FP8 E4M3 KV and 0.0239 with FP16 KV on the same inputs and positions; top-1 agreement is 97.4%.

KL follows the content rather than position alone, and no difference is attributable to the FP8 KV cache up to 128K. The English rise after its opening pages appears with FP16 KV as well. Three documents are not a broad long-context benchmark, and the SQLite source is unusually predictable.

Serving check — 2026-10-16

Before release, the checkpoint passed a functional regression check in the configuration above: concurrent long-context requests, image and video inputs, MTP speculative decoding with the unchanged mtp-fp8/ draft, and needle retrieval at 211K prompt tokens (5/5). This confirms loading and serving behavior only; it is not a capability benchmark.

Previously published GSM8K, HumanEval, MBPP, IFEval and AIME results describe the previous expert weights. They have not been rerun on this release and are not claimed for it. The historical card remains available at repository revision b2aea8e9a167253426f5cd15fc22769671e5abda.

Notes and limitations

• The same-mother BF16 comparison is bounded to the text subset and approximation above; no universal BF16-to-AWQ capability-loss figure is claimed.
• This quantization and its reported runtime checks do not certify every production, agent, long-output or multimodal workload.
• The upstream “Uncensored” name identifies the mother model; it does not guarantee refusal behavior, safety or suitability for a particular application.
• Preserve the bundled K/V scales; they are reused historical calibration, not newly fitted scales for these experts.
• This derivative uses the included Qwen Community License 1.0.

Integrity

Run sha256sum -c SHA256SUMS after downloading. The checksum list covers every file in this root release package except itself: all indexed shards, tokenizer/processor assets, config, license, this card and public metadata. It excludes the separately preserved mtp-fp8/ directory, which has its own checksum manifest.

Optional FP8 MTP

The existing optional draft in mtp-fp8/ is unchanged by this release. It stores only MTP routed experts in FP8 E4M3 with explicit block scales; other draft parameters retain source precision. Select that directory explicitly as the speculative draft with modelopt_mixed in a compatible Qwen4Exp MTP runtime. It retains its own K/V scales and documentation. A full unfiltered repository download also includes its 4.88 GiB tensor payload.

The serving check above used this unchanged draft with the new target weights. The BF16 distribution comparison covers the target model only, not the draft. The draft's earlier bounded checks remain documented in its directory.