OrcaRouter Qwen3.8-Flash-Next-Uncensored — NVFP4 (GPTQ) for Strata

[!WARNING] Withdrawn from Strata NVFP4's installer (2026-10-10): as an agent, this abliterated build is much worse than the original Qwen.

• A multi-step Claude Code task (building an interactive 3D page), same prompt on both models: the original Qwen3.8-Flash-Next (our GPTQ of it) ran 175 steps, checking and fixing its work from screenshots, and gave a clearly better result. This build stopped after 10-70 steps (3 runs) and declared the work done.
• It closes its thinking sooner. Over 7,869 positions of the original model's thinking (teacher-forced), this pack raises the probability of </think> by +0.12 nats on average.
  • On a captured 72K-token Claude Code request (5 greedy runs each) it thought 997 tokens on average, the original 1,699.
  • Its style of thinking is the same; it just stops earlier.
  • The likely cause is OrcaRouter's weight edit: it removes the refusal direction from all 149 residual-writing matrices of every layer, including the late layers whose edit pushes </think>.
• What the installer offers instead. From Strata NVFP4 0.1.41-nvfp4.5:
  • the original Qwen (--family qwen-nvfp4-gptq), with an opt-in switch (--uncensored on, off by default);
  • the switch removes refusals at run time, in layers 8-33 only, and leaves the weights untouched;
  • with thinking on it refused 0 of 104 English and 0 of 30 Ukrainian held-out harmful requests;
  • </think> +0.02 nats; thinking length as the original's.

An existing install of this model keeps working, and the files here are unchanged.

OrcaRouter's abliterated Qwen3.8-Flash-Next with every routed expert in NVFP4, re-quantized from the BF16 checkpoint by GPTQ, packed for Strata NVFP4: the 125B hybrid MoE on one RTX 20-50 card (12 GB of VRAM or more) and 64 GB of RAM or more.

Installer: Strata NVFP4 v0.1.41-nvfp4.1 to .4 listed this repository as --family orca-nvfp4 (the default from .3). From v0.1.41-nvfp4.5 the installer no longer offers it (see the warning above). To run this model by hand, see Run it.

• Chain: Qwen/Qwen3.8-Flash-Next → orcarouter/Qwen3.8-Flash-Next-Uncensored (abliteration, BF16) → this quantization.
• Same size and the same decode cost as the ModelOpt NVFP4 experts (63.3 GiB, 4.5 bits a weight), about 33% of their error on held-out tokens: the same format, other codes and scales.
• End to end, an earlier build of the same recipe (2026-10-02, the fork's docs/NVFP4.md) had a median per-answer KL of 0.0070 against an all-Q8_0 reference where ModelOpt's experts had 0.0107 (65%), better in 12 of 12 prompts. This build was checked as described under Checks, not re-measured that way.
• Only for Strata NVFP4. The files are Strata's own runtime formats (an expert pack, GGUFs the fork reads, the MTP draft head). They do not load in transformers, vLLM or llama.cpp.

The model does not refuse. It is an abliterated fine-tune (the refusal direction was removed from the weights; see the base model's card). It will follow harmful requests. What it is used for is on whoever runs it.

Files

Which parts depend on the abliteration:

• Model-specific: the experts (down_proj was edited in every layer; gate_up_proj is Qwen's, but GPTQ fits the codes to this model's activations), the dense GGUF (o_proj, out_proj, the shared experts' down_proj, ple.value_proj were edited), pack/dense.bin, the token embedding (edited) and the MTP head.
• The same as Qwen's original: the n-gram table in ple-fp8.gguf (the abliteration does not touch it; 31 of the 33 checkpoint shards that hold it are byte-identical to Qwen/Qwen3.8-Flash-Next, the other two also hold edited tensors), lm_head, the tokenizer, the chat template and the license.

The dense GGUF is the converted ModelOpt checkpoint's GGUF (jpezzulli/OrcaRouter-Qwen3.8-Flash-Next-Uncensored-ModelOpt-NVFP4, whose dense weights, PLE table, embedding and MTP head are OrcaRouter's BF16/FP8 tensors, excluded from quantization) with its 144 routed-expert tensors (63.3 GiB) left out: next to pack/experts.bin the engine never reads them. Every remaining tensor and every metadata key is byte-identical to that GGUF, and the engine's output with it is bit-identical (below).

How it was quantized

NVFP4 stores a weight as a 4-bit E2M1 code times its 16-value block's FP8 (E4M3) scale times one FP32 scale per expert matrix. ModelOpt's NVFP4 scales each block to its largest value and rounds every weight to the nearest code. Here each expert was re-quantized from the BF16 checkpoint with tools/requant.py --method gptq from the fork:

• Calibration: the engine's own MoE inputs at every layer (STRATA_DUMP_MOE_INPUT, STRATA_DUMP_MOE_LAYER=all) for 57.5K tokens: Ukrainian and English Wikipedia, llama.cpp and Rust sources, and 16 of the model's own chat answers (data/requant_calib in the fork), with the routing of each token.
• Hessians: for each expert, H = sum w^2 x x^T over the tokens routed to it (w its routing weight: its output enters the residual times w), mixed with the layer's average input as 256 pseudo-tokens so rarely used experts get a sensible one. down's Hessian is built from silu(gate x) * up x of the already quantized gate/up, so down corrects the error gate/up left.
• GPTQ: columns left to right in blocks of 128, 1% damping; each column's rounding error is spread over the columns not yet quantized through the inverse Hessian.
• Block scales: each 16-column group's FP8 scale is chosen on the weights as GPTQ has updated them, among the codes for amax -> 6 and amax -> 4 ("four over six") and two neighbours of each, by the least squared error weighted by diag(H). The FP32 scale per expert matrix leaves 25% headroom so the updates do not saturate E4M3.
• Every routed expert is NVFP4 (no 8-bit layers): the pack is as large as ModelOpt's and a decode step costs the same. The shared experts, attention and DeltaNet projections are Q8_0 in the GGUF, as in the ModelOpt conversion.

Error against BF16

requant.py analyze: per layer, on held-out tokens (every 4th calibration token, not used for the Hessians), the routed-weighted squared error of the experts' whole output down(silu(gate x) * up x) against BF16, relative to the output's energy; summed over the 48 layers weighted by each layer's energy:

48 layers, 57551 tokens per layer (14387 held out)

GPTQ below RTN in 48 of 48 layers; ratio GPTQ/RTN per layer: min 0.21, median 0.39, max 0.46

Per layer

Checks

Strata NVFP4 0.1.40.3 on an RTX 5090 (32 GB) with 128 GB of RAM, the arguments under Run it (--spec 6 --spec-min-p 0.7), the pack read from a second NVMe drive.

• It answers. "Explain in detail how a hash table works ... at least 300 words" (no thinking): a correct 493-token answer that ends by itself (EOS), decoded at 188.7 tok/s (2.37 tokens per speculative round, 12.6 ms a round). A 2,100-token prompt (a CUDA source file cut off mid-line, with no question): prefill 2,763 tok/s; with --stop-eos the model completes the cut identifier (q2[l -> q2[l0 /, as the file's pattern goes) and ends the user turn after 4 tokens. The same prompt run for 200 tokens without --stop-eos (the deterministic runs below) goes on into the assistant's turn, whose thinking correctly describes what the file implements (CUDA dot products for the q2_0 / IQ2 formats, taken from llama.cpp).

• The dense-only GGUF changes nothing. orca-nvfp4-dense.gguf against the full GGUF it was cut from: the KV section (metadata, tokenizer) and all 1,367 kept tensors are byte-identical, the 144 dropped tensors are exactly blk.*.ffn_{gate,up,down}_exps. With a fixed expert cache (--expert-cache 6000) and deterministic settings, 200 greedy tokens after the 2K prompt and the first token's full logits are bit-identical with either GGUF.

• A decode round costs the same as with ModelOpt's experts. Three interleaved pairs on the hash-table question, each answer to its end, expert cache auto (8,343 slots in every run), against ModelOpt's pack (the same GGUF, PLE, embedding and MTP head; only experts.bin differs):

  experts	ms per round	tokens per round	tok/s
  ModelOpt NVFP4	12.55 ± 0.13	2.40 ± 0.05	191.6 ± 1.7
  this GPTQ pack	12.55 ± 0.05	2.33 ± 0.01	185.6 ± 0.8

  A round (one forward of the main model plus the drafts) costs exactly the same. The tokens per round depend on how many MTP drafts each answer's text lets through; in these three pairs they were 3% fewer with this pack (paired difference -0.075 ± 0.042), which three pairs cannot tell from the answers' own spread.

Run it

Build or download Strata NVFP4 (Windows, NVIDIA driver 580+; see its README for the requirements: RAM, pagefile, large pages). Then:

hf download Maximilian228/OrcaRouter-Qwen3.8-Flash-Next-Uncensored-NVFP4-GPTQ-Strata --local-dir models\orca-gptq

One-shot (prompt.txt holds token ids):

strata.exe --pack models\orca-gptq\pack ^
  --native models\orca-gptq\orca-nvfp4-dense.gguf --native-dense-gguf models\orca-gptq\orca-nvfp4-dense.gguf ^
  --ple-gguf models\orca-gptq\ple-fp8.gguf --embd-gguf models\orca-gptq\token-embd-bf16.gguf ^
  --mtp models\orca-gptq\mtp --spec 6 --spec-min-p 0.7 --prefill auto ^
  --expert-profile data\expert-profile.bin --expert-cache auto ^
  --max-context 262144 --kv int8 --tokens-file prompt.txt --max-new 512 --stop-eos

--max-context 262144 fits a 32 GB card; with less VRAM lower it (the fork's README and setup use these):

data\expert-profile.bin ships with the engine. As a server (OpenAI and Anthropic APIs), put the same arguments in the release's config\strata-nvfp4.json ("args") and point "tokenizer" at models/orca-gptq/pack/tokenizer. Pictures need the image encoder (mmproj-f32.gguf, not in this repo: the release's prepare-model.cmd builds it, or convert_hf_to_gguf.py --mmproj on the BF16 checkpoint); point the config's "vision" entry's "model" at orca-nvfp4-dense.gguf (the encoder reads only its vocabulary). Without the encoder, drop --vision from the arguments.

Already have the ModelOpt files from the release's prepare-model.cmd? Its ple-fp8.gguf and token-embd-bf16.gguf hold the same tensors; skip them with --exclude ple-fp8.gguf --exclude token-embd-bf16.gguf and point the flags at yours.

Reproduce

With the fork's tools and the BF16 checkpoint (hf download orcarouter/Qwen3.8-Flash-Next-Uncensored, 360 GB = 335 GiB; the repository is gated with automatic approval, so accept its terms on its page and hf auth login first), starting from the ModelOpt pack (packs\orca-nvfp4, the README's "Prepare the model"):

set STRATA_DUMP_MOE_INPUT=calib\d
set STRATA_DUMP_MOE_LAYER=all
strata.exe <the arguments above, --pack packs\orca-nvfp4> --tokens-file data\requant_calib\calib_uk.txt --max-new 1   (and en, code, chat)
.venv\Scripts\python tools\requant.py analyze --bf16 models\orca-bf16 --calib calib\d --out calib\errors.jsonl
echo {"default": {"gu": "nvfp4", "d": "nvfp4"}} > calib\plan_nvfp4.json
.venv\Scripts\python tools\requant.py pack --bf16 models\orca-bf16 --calib calib\d --plan calib\plan_nvfp4.json ^
  --method gptq --base packs\orca-nvfp4 --out packs\orca-nvfp4-gptq4

The dump for this upload was taken with the engine running a GPTQ pack of the same model with Q8_0 down in 27 layers (packs\orca-nvfp4-gptq-q8d, the README's optional pack: the closest to BF16 there was), not with ModelOpt's pack: the inputs of layer N depend on how layers 0..N-1 were quantized, and the closer they are to BF16, the closer the Hessians are to the BF16 model's. A dump with ModelOpt's pack (the commands above) gives slightly different Hessians and so slightly different codes, not a different method. The same analysis on the 2026-10-01 dump (from an all-Q8_0 reference pack) gave the same picture: scale search 61.7%, GPTQ 33.5% of RTN (here 61.6% and 33.4%).

License

The Qwen Community License 1.0, as Qwen/Qwen3.8-Flash-Next and orcarouter/Qwen3.8-Flash-Next-Uncensored ship it (the same LICENSE file; the OrcaRouter card's front matter says Apache 2.0, its LICENSE file does not). Strata NVFP4 itself is MIT.