RVN Qwen3.8 Flash-Next — W4A16 NVFP4

This is the NVFP4 format companion to 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored, derived from exact source revision 38b6e951065008472ebfb72d1736c9df64bb0e9a. It is not a new training or abliteration run.

Why this release exists

This NVFP4 export was made at the request of our community member @Hector111 in “Could you please provide the NVFP4 version” — discussion #1:

Could you please provide the NVFP4 version?

Thanks to Hector111 for the request, and to everyone who expressed interest in an NVFP4 release.

Artifact

• Architecture: Qwen4ExpForCausalLM / qwen4_exp_text; main CausalLM only, with no active MTP head.
• Exact model index SHA-256: 8aa43a8ca030f4c31e1920a61736e43d0d844b6fc51b9472759c0dad71dd924d (35,726,105 bytes).
• 242 indexed safetensors shards, 296,110 tensors, and 180,244,203,544 tensor-payload bytes (180,283,328,728 on-disk shard bytes including safetensors headers).
• All 242 shard hashes were independently checked. The 1,198 non-expert tensors were retained byte-identically.
• Expert matrices use W4A16_NVFP4: packed NVFP4 E2M1 weights, FP8 E4M3 scales with group size 16, FP32 second-level scales, BF16 activations, and the Marlin backend in the tested runtime.
• Scalar input_scale=1 fields are unused W4A16 loader slots, not measured activation calibration. This is not W4A4, not calibrated ModelOpt, and not all weights are 4-bit.
• The large PLE tensor remains in its original retained shard; it was not needlessly reshared.
• Published chat_template.jinja SHA-256 c3cf9e34abf4f9e36c2d72165aa9c132d3e2a725b6c2586aaa3a8af9d7a81041 matches the canonical source template. Its presence is cryptographically checked during staging; this statement alone is not a native-chat runtime result.

Release quality gate

The gate used the full PyTorch-example WikiText-2 test split (not the raw variant): 603 disjoint 512-token windows and 308,104 scored next-token targets.

This is an owner-approved release with a disclosed strict-format limitation: the predeclared 90% operational threshold was NOT passed. The threshold and failed gate remain unchanged; successful serving and small PPL drift are separate findings. The operational set contains six categories with eight cases each. The 48-case messages were rendered once from the canonical source chat template with enable_thinking=False into token IDs under wire-contract SHA-256 31ecda676d2c306fe24bf65719b5f75d662af0589529e1186ab810cdafc63bc4, then submitted through /v1/completions with seed 0, temperature 0, maximum 256 generated tokens, and stop tokens im_end and endoftext. This is not a native-chat-template retention test; candidate native-chat behavior requires a freshly loaded server and is a separate check. These are bounded release checks. No full-distribution KL was measured, and the result does not establish universal capability preservation, long-context quality, throughput, or safety.

The BF16 source reference exposed 145.31 GiB of expert tensors through PLE/UVA CPU offload and used PYTORCH_ALLOC_CONF=expandable_segments:False,pinned_max_round_threshold_mb:128. Its isolated one-buffer host-staging correction reduced transient host duplication and did not change checkpoint tensors; three real BF16 canaries passed before the full reference measurement. The BF16 checkpoint was not fully resident on the 95 GB GPU.

Full-GPU and native FP4 follow-up

The same released W4A16 weights were also tested on 2 x NVIDIA RTX PRO 6000 Blackwell Workstation Edition (97,887 MiB each) with tensor parallel 2 and no PLE CPU offload. Measured loading was 84.97 GiB per rank; sampled peak use was approximately 90.48 GiB/GPU in that run. Three direct chat canaries passed. This does not mean the entire model fits on a single 16/24/32 GB GPU.

All rows used 603 windows / 308,104 scored target tokens and the same 48-case contract; degenerate outputs were zero. Historical BF16 and TP1 results above are separate runs. The BF16 reference and all candidate runs remained below the predeclared 90% strict-format threshold. Failures included fenced JSON, extra explanation and exact case matching; these limitations are disclosed rather than counted as passes. The small PPL changes are not percentages of intelligence lost.

Actual loopback streaming, with concurrency 1, eager execution, input lengths 512/1536, output length 128, seeds 17/23 and eight measured requests per cell, gave 25.59 output tokens/s for W4A16/Marlin versus 23.55 for uncalibrated W4A4/CUTLASS. There were 32 requests / 4,096 output tokens per arm. Separate profiler traces proved native SM120 E2M1 GEMMs on both ranks, not Marlin fallback. The native experiment was about 8% slower under these specific settings; this is not a claim about calibrated NVFP4, B12x, other hardware, long context or batched throughput. flashinfer_b12x was source-audited but not benchmarked.

The published artifact remains W4A16_NVFP4. No calibrated activation scales or new W4A4 checkpoint are being released. See paired measurement evidence and full-GPU evidence.

For the tested two-GPU setup, use the pinned runtime below with VLLM_PLE_CPU_OFFLOAD=0, --tensor-parallel-size 2, --moe-backend marlin, --max-model-len 2048, --max-num-seqs 1, --max-num-batched-tokens 512, --kv-cache-memory-bytes 1073741824, --no-enable-prefix-caching, and the other shared serving flags below. Use the included checkpoint as --model; do not change its quantization metadata. The one-GPU setup instead requires substantial host RAM and the PLE CPU-offload settings below.

Download

hf download 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-NVFP4 --local-dir ./rvn-flash-next-nvfp4

Tested serving scope

Actual short English, Turkish, JSON, and parsed tool-call smoke checks passed using a single NVIDIA RTX PRO 6000 Blackwell Server Edition (95 GB), PLE CPU offload, BF16 activations, Marlin, eager execution, concurrency 1, and maximum context 2,048. The model was not fully resident on that GPU; PLE was CPU-offloaded.

The tested immutable runtime was docker.io/vllm/vllm-openai@sha256:fc120ece0a388cc0aa1caad4a9f1cd92113484ab7ec2fd0efadd62585be05bf8 with vLLM 0.1.dev20073+g8e685d198, Transformers 5.15.1, and Torch 2.13.0+cu130. This is a pinned custom-runtime compatibility result, not a stock Transformers loading claim.

The image registers Qwen4ExpForCausalLM to the same implementation as Qwen3_8FlashNextForCausalLM, but its PLE validator omitted that registry-identical alias. Only the following one-line allowlist correction was used:

 if self.model_config.architecture not in {
+            "Qwen4ExpForCausalLM",
             "Qwen3_8FlashNextForCausalLM",

Apply that correction only when the original vllm/v1/worker/gpu_worker.py SHA-256 is 923767cea120a027ca36683bd48b7659eadbfa4cbd4acbcf52a80cee0c0a0ec4; the exact corrected file SHA-256 is aa1d93672b32baf78551aeab7bbf6cbc3caaec00fd0263c3b45be725a65a68db. Do not broaden the PLE allowlist. The published config also preserves the original sparse-attention parameters while translating the source enum for this pinned runtime, and records zero MTP layers because the exact source index contains no MTP tensors.

The tested server additionally set VLLM_PLE_CPU_OFFLOAD=1, VLLM_PLE_OFFLOAD_READY_TIMEOUT=1800, PYTORCH_ALLOC_CONF=expandable_segments:False, and VLLM_WORKER_MULTIPROC_METHOD=spawn, then used --dtype bfloat16 --moe-backend marlin --enforce-eager --distributed-executor-backend mp --disable-custom-all-reduce --mamba-ssm-cache-dtype float32.

Limits and responsibility

This abliterated, uncensored model is not a safety product. It may produce inaccurate, offensive, explicit, unsafe, or unlawful material. Operators are responsible for lawful use, their own risk assessment, access controls, moderation, monitoring, and output verification. Do not treat outputs as professional advice or verified facts.

The quality evaluation is limited to the disclosed PPL windows and 48 operational probes. No broad task suite, full-distribution KL, long-context, multi-user throughput, or comprehensive safety evaluation is claimed for this NVFP4 artifact. The bounded single-user streaming measurements are disclosed above.

Use is subject to the Qwen Community License 1.0 at the pinned source link in the model-card metadata and to applicable law.

Evidence

publication-manifest.json records every uploaded path, byte size, and SHA-256. The evidence/ directory contains the release summary, privacy-sanitized export and independent-verification receipts, and only the additional files explicitly allowlisted by the signed-off quality summary.