Qwen3.8-27B QUASAR NVFP4 for NInfer

Native NInfer v3 artifact built from QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4, an NVFP4 quantization-aware-trained version of Qwen/Qwen3.8-27B. It includes Text, Vision, MTP, DFlash2, the optimized proposal head, and frontend resources. No separate draft-model download is needed at inference time.

Use the mobile NInfer fork, validated at b53468fa. This file is specific to NInfer; it is not a Transformers checkpoint, Safetensors distribution, GGUF file, or generic NVFP4 interchange format.

V3 release and compatibility

The September 27, 2026 release upgrades the previously published v2 container to v3. All 1,328 tensor payloads are preserved byte-for-byte. No weights were requantized. Five frontend resources are also unchanged; the chat template is replaced by the maintained tools/chat_templates/qwen3_8.jinja. V3 adds explicit configurations, logical bindings and per-input activation permissions to the container metadata.

This is a format migration, not a new QAT checkpoint or a claim of better model quality. The updated template can change rendered prompts, and newer runtime arithmetic can change generated continuations. Weight preservation does not imply output identity across releases.

• main and tag v3 distribute the v3 artifact for the current runtime.
• Tag v2 preserves the previous artifact, model card, checksum and runtime instructions.
• Current NInfer loads v3; the old v2 runtime cannot load this file.
• The public filename remains qwen3_8_27b_nvfp4.ninfer.

Artifact

See artifact-manifest.json for current metadata and qwen3_8_27b_nvfp4.ninfer.upgrade.json for migration verification. The unmodified qwen3_8_27b_nvfp4.v2.conversion.json records the original v2 conversion; its artifact size, recipe and runtime identifiers are historical, not the v3 schema or current runtime configuration.

Requirements and download

• 64-bit Linux and an NVIDIA Blackwell GPU with FP4 support;
• CUDA Toolkit 13.1, CMake 3.28+, and the dependencies described in the runtime README;
• the validated NInfer revision below or a compatible newer v3 runtime.

Validation used an RTX 5090 Laptop GPU with 24 GiB. Artifact size is not resident GPU memory: the selected speculative backend, KV format/capacity, concurrency, Vision and CUDA Graphs affect memory use. Inactive MTP/DFlash2 weights are not uploaded for another selected backend.

Build the validated runtime:

git clone https://github.com/MirkoCovizzi/ninfer-rtx5090-mobile.git
cd ninfer-rtx5090-mobile
git checkout b53468fae6af160c3458bbe87e157b6201a4a12b
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build -j

Download the pinned v3 release and verify its checksum:

hf download MirkoCovizzi/Qwen3.8-27B-QUASAR-NVFP4-NInfer \
  qwen3_8_27b_nvfp4.ninfer SHA256SUMS \
  --revision v3 --local-dir models/quasar-v3
(cd models/quasar-v3 && sha256sum --check SHA256SUMS)

For the historical artifact, use --revision v2 and the runtime instructions on that tag's model card.

Run

MTP with KVarN KV cache:

./build/apps/ninfer models/quasar-v3/qwen3_8_27b_nvfp4.ninfer \
  --prompt "Explain quantization-aware training in three sentences." \
  --max-context 4096 --max-new 256 \
  --kv-dtype kvarn --spec mtp --draft-tokens 3 --lm-head-draft

OpenAI/Anthropic-compatible server with DFlash2:

./build/apps/ninfer-serve models/quasar-v3/qwen3_8_27b_nvfp4.ninfer \
  --host 127.0.0.1 --port 18080 \
  --max-concurrency 1 --max-context 8192 --kv-capacity 8192 \
  --prefill-chunk 1024 --kv-dtype int8 \
  --spec dflash2 --draft-tokens 7 --lm-head-draft

For ordinary decoding, omit the three speculative options. MTP accepts 1–5 draft tokens; DFlash2 accepts 1–15. The best depth depends on the workload. Speculation is selected at startup, not by an OpenAI/Anthropic request field. --lm-head-draft selects the optimized proposal head. CUDA Graph decode and compatible-prefix reuse are enabled by default. Add --vision to the server for image/video requests. Runtime KV choices are BF16, INT8, FP8, NVFP4, K8V4 and KVarN; the validation scope for each route is recorded below.

NInfer uses one GPU with startup-fixed concurrency of 1–8 requests. It does not provide CPU/GPU offload, multi-GPU execution or large-scale preemptive continuous batching.

Weight provenance and numerical boundaries

The original conversion preserved the QUASAR source's packed E2M1 codes, E4M3FN block scales, FP32 weight divisors and activation divisors for 400 of its 496 Text Linear matrices. Fusing compatible source groups produced 256 NVFP4 parents: 32 attention, 96 GDN and 128 MLP parents. Fused groups had bit-identical global-scale words; they were not rescaled or requantized.

The 96 narrow GDN control projections (in_proj_a / in_proj_b, output width 48) crossed the original conversion's explicit BF16 boundary:

W[n,k] = decode_e2m1(code[n,k]) * decode_e4m3fn(scale[n,floor(k/16)]) / weight_global_scale

Represented values were rounded to BF16 and fused A-then-B into [96,5120] parents. These are decoded QUASAR controls, not substitutions from the pre-QAT official BF16 checkpoint. The v3 upgrade introduces no further weight casts.

QUASAR supplies the BF16 MTP source. The pinned official Qwen checkpoint supplies the Q8 vocabulary endpoints, Vision and optimized proposal-head source data. Existing frontend resources come from that conversion, except for the maintained template installed during the v3 upgrade.

DFlash2 originates from Inco AI, obtained through the pinned z-lab mirror. Its 81 source BF16 tensors were converted to 66 stored tensors: 21 Q8 matrices and 45 BF16 normalization, convolution and selector tensors. The companion is Q8/BF16, not QUASAR NVFP4. No additional training or QAT of DFlash2 was performed for this distribution.

Reproduce the upgrade

From the runtime checkout above, using Python 3.11:

hf download MirkoCovizzi/Qwen3.8-27B-QUASAR-NVFP4-NInfer \
  qwen3_8_27b_nvfp4.ninfer SHA256SUMS \
  --revision v2 --local-dir models/quasar-v2
(cd models/quasar-v2 && sha256sum --check SHA256SUMS)
python3.11 tools/upgrade_ninfer_v2_to_v3.py \
  models/quasar-v2/qwen3_8_27b_nvfp4.ninfer \
  models/quasar-local-v3/qwen3_8_27b_nvfp4.ninfer
python3.11 -m tools.artifact.inspect \
  models/quasar-local-v3/qwen3_8_27b_nvfp4.ninfer --json

The upgrader is CPU-only and uses the Python standard library; the output must be a new path. It assigns a fresh artifact UUID, so a locally regenerated file has a different whole-file checksum even when its weight payloads and template match. SHA256SUMS applies to the published download. The original source-conversion commands and pinned inputs remain on the v2 tag.

Validation and limits

The exact published v3 file was used for integration validation on RTX 5090 Laptop (24 GiB), driver 595.91.07, CUDA 13.1.115, GCC 15.3, Release/sm_120a and Python 3.11.16. The integration was published as runtime b53468fa in PR #37. The full build and 121 self-contained CTest checks passed after integration fixes. Selected real-artifact checks additionally passed:

• loading/binding of Vision, MTP and optimized proposal-head resources;
• prefill state/logit equality across cache policies, tool-prefix reuse and token agreement;
• causal-scoring tiles, overlapping suffixes and repeated-window isolation;
• ordinary/MTP3 greedy parity at C=1 across all six KV formats, 512 outputs per case;
• ordinary/MTP5 greedy parity at C=8 for KVarN and BF16, ragged budgets of 128–121 outputs;
• MTP5 at C=2 with unequal restored KVarN prefixes, budgets 128/127;
• DFlash2 K15, C=2, INT8, CUDA Graphs, optimized head and checkpoint/state checks;
• KVarN MTP5 Vision stop/resume, media continuation and actual Host state restoration.

Release inspection confirmed v3 framing, the declared object encodings, all 1,328 tensor payloads and five unchanged resources against the published v2 file, and exact equality of the replacement template with the maintained template. The artifact's complete SHA-256 is recorded above.

Greedy parity is against ordinary decoding in the same runtime, not across runtime versions. These checks do not establish broad reasoning/retrieval quality or full-corpus perplexity. Source-model quality scores are not measurements of this converted artifact. The ongoing before/after performance comparison is not complete; this release makes no new throughput or quality claim. Historical v2 performance remains on that tag, with its original conditions.

Provenance

The historical conversion report records HEAD 0f54b98e because conversion ran with integration changes before they were committed as d4bc75db. The original manifest on tag v2 records that distinction. The v3 migration preserves that original report rather than rewriting its history.

License and credit

Apache-2.0, matching the official Qwen3.8, QUASAR and pinned DFlash2 source model cards. Credit belongs to Qwen for the base model, the QUASAR authors for QAT, Inco AI and the DFlash authors for DFlash2, and NInfer and its contributors for conversion and inference.

• QUASAR paper
• DFlash 2 announcement
• Upstream NInfer