Qwen3.8-Flash-Next NVFP4 (LIL) with BF16 PLE tables

A derivative of local-inference-lab/Qwen3.8-Flash-Next-NVFP4 at revision 6c01ac378d22f6652c9f392c87700496eca8e5dc, itself a quantization-aware-distilled build of Qwen/Qwen3.8-Flash-Next. Public, ungated. No retraining was done.

What changed

PLE tables = LIL's QAD-trained NVFP4 tables dequantized to BF16 (no retraining, no other weights changed).

• The 128 n-gram (PLE) tables model.language_model.layers.1.ple.ple_embedding.ngram_embedding.shard_{0..127}.weight were dequantized from NVFP4 (E2M1 values x E4M3 block scale per 16 elements x FP32 global weight_scale_2 = 3.3242362e-05) in FP32 and rounded to BF16 once. They are stored as BF16 [2500012, 160] in ple-bf16-00..42.safetensors (3 tables per file, 102.4 GB total). The .weight_scale tensors and weight_scale_2 are removed.
• config.json: text_config.ple_embedding_dtype nvfp4 -> bfloat16; the PLE entry removed from quantization_config.quantized_layers. hf_quant_config.json: the same entry removed.
• model.safetensors.index.json regenerated for the new layout. model-00026-of-00041.safetensors was rewritten without the PLE tensors (its three small ple.norm_* tensors are copied byte-for-byte); model-00027..00041 were dropped because they contained only PLE tensors.
• Every other tensor and file is byte-identical to the source revision, including attention and shared-expert MXFP8, routed-expert NVFP4 and the MTP layer.

Why: runtimes that load the PLE table to host memory only for BF16 (for example vLLM's PLE CPU-offload path) cannot load NVFP4 PLE tables.

Rounding footnote: BF16 rounding of the FP32 dequant has max relative error 0.00307 and mean 0.0013 on sampled rows (bound 2^-8 = 0.0039). Against the original unquantized BF16 PLE table, a 400-row sample of shards 0-2 gives cosine ~0.995 and relative L2 ~0.097; that difference is the NVFP4 quantization the LIL model was trained with, not a conversion error. This repo does not use the original table.

Not validated end to end here: no perplexity or generation test of this exact checkpoint has been run by the publisher of this derivative. Note that export-manifest.json and HYBRID.json are carried over from the source unchanged and still describe the source's NVFP4 PLE.

Serving note (vLLM)

With vLLM 0.31's quantization resolver (checked on CPU against this repo's config.json / hf_quant_config.json, with LIL @6c01ac37 as the control), the PLE table resolves to the unquantized embedding method, i.e. BF16 tables that can be held in pinned host memory with the PLE CPU-offload mode (--engram-config cpu_offload); the MTP draft experts (mtp.layers.*.mlp.experts) resolve to W4A16_NVFP4 and the routed experts to NVFP4, unchanged from the source. Expect about 95 GB of host RAM for the BF16 tables. This resolver check is not a full serving benchmark; no quality or throughput numbers are claimed for this exact checkpoint.

Provenance and license

• Source: local-inference-lab/Qwen3.8-Flash-Next-NVFP4 @ 6c01ac378d22f6652c9f392c87700496eca8e5dc (all LFS files sha256-checked against the Hub before use).
• Upstream base: Qwen/Qwen3.8-Flash-Next. License: Qwen Community License 1.0, carried over (LICENSE). See the source card for training details and its own terms.
• Built and uploaded by btbtyler09.