bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4

NVFP4 quantization of bottlecapai/ThinkingCap-Qwen3.8-Flash-Next, the ThinkingCap finetune of Qwen3.8-Flash-Next (Qwen Team, 2026) that reasons in fewer tokens.

➡️ Full model description, evaluation results, recommended sampling params, and citation: see the main model card at bottlecapai/ThinkingCap-Qwen3.8-Flash-Next.

About this quantization

What is stored at which precision:

• the routed experts, in every layer and in the MTP head, in NVIDIA's NVFP4 format: 4-bit E2M1 values with an FP8 scale per 16 weights and one FP32 scale per expert projection, stored in the compressed-tensors nvfp4-pack-quantized format. Activations stay in bf16 (weight-only, NVFP4A16), and no calibration data was used. An expert's gate and up projections share one FP32 scale, because vLLM runs them as one matrix multiply;
• the n-gram embedding table in FP8 (E4M3) with one per-tensor scale, about 51 GB instead of 102 GB in bf16;
• everything else in bf16: attention and linear attention, the shared experts and router, the vision tower, embeddings and lm_head.

≈132 GB instead of ≈360 GB in bf16.

Publish-time checks

• Structure: the shard index is complete; the MTP layer (4,637 tensors) and the vision tower (333 tensors) are kept; on 45 sampled experts, including the MTP head's, the gate and up projections share one FP32 scale.

• Serving smoke test on two RTX PRO 6000 Blackwell GPUs (tensor parallel 2, vLLM 0.31.0), loading straight from this repo: the model serves, answers an image question correctly (the vision tower works) and the reasoning parser separates the thinking. vLLM runs the weight-only 4-bit experts through its Marlin kernel.

• Accuracy and thinking length against the bf16 model, served on two H200s (tensor parallel 2, vLLM 0.31.0, no speculative decoding). Seeds are averaged within a question, and the 95% interval comes from resampling questions:

Accuracy is unchanged within noise; on AIME 2026 the NVFP4 model thinks 18% longer than the bf16 model, mostly on the longest problems. No answer hit the token cap.

• MTP speculative decoding on the same two H200s (3 draft tokens, probabilistic drafting, AIME 2026 and RealWorldQA): 2.76 tokens accepted per decoding step on average, as for the bf16 and FP8 models (2.77). vLLM issue #55357, MTP draft acceptance dropping to zero with repetition inside the thinking on this architecture, is still open; it did not reproduce in our test.

Usage (vLLM)

Requires vLLM 0.31.0 or newer.

vllm serve bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4 --tensor-parallel-size 2 --max-model-len 262144 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml

Use the sampling settings from the main model card. vLLM keeps the n-gram table in pinned host RAM by default (about 51 GB). The checkpoint keeps the MTP head; to serve with MTP speculative decoding (measured under Publish-time checks):

vllm serve bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4 --tensor-parallel-size 2 --max-model-len 262144 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3,"draft_sample_method":"probabilistic"}'

License

Released under the base model's own license, the Qwen Community License 1.0 (full text in LICENSE). Its conditions carry over to this model, including the separate license from Qwen that Model-as-a-Service and AI work-assistant businesses need for commercial use.