ThinkingCap: Qwen3.8-Flash-Next

ThinkingCap brings our shorter-thinking post-training to Qwen3.8-Flash-Next (Qwen Team, 2026), a mixture-of-experts model with image input. ThinkingCap Qwen3.8-Flash-Next cuts reasoning tokens by 38.5% on average (22% to 51% depending on the benchmark) at an average accuracy of 88.8% against the base model's 89.8%. Check our blogpost for more details.

The base model and ThinkingCap answering the same prompts side by side, with each answer's token count and time; playback is sped up. Also in 4K.

Token efficiency and benchmark performance

Evaluation details

Models

Base Qwen/Qwen3.8-Flash-Next against bottlecapai/ThinkingCap-Qwen3.8-Flash-Next, shown as Ours.

Metrics

• Accuracy (Base / Ours) — fraction of correct answers. The τ²-bench score is an unweighted mean over its three domains (airline, retail, telecom), not pooled over tasks.
• Thinking tokens (Base / Ours) — mean length of the <think> trace, answer excluded. τ²-bench is a multi-turn agentic episode, so for that row the count is the reasoning summed over every turn of the episode (about 15 turns on average), not a single trace.
• (Thinking token) Reduction — the relative change between the two mean columns beside it, (Ours − Base) / Base, where each mean is taken over every question and seed of the benchmark. A saving is shown as a green ↓ percentage.
• Macro average (bottom row) — equal-weight mean across benchmarks of each column, including the reduction: it is the mean of the per-benchmark reductions, not the ratio of the two token figures beside it.

We separately track truncation: on the single-turn benchmarks, the <think> trace never closes because the model hits the generation cap or stops while still reasoning, so no answer is produced; τ²-bench marks an episode if any of its turns hits the per-turn cap. Equal-weight across benchmarks it is 0.05% → 0.00%.

Serving

Hardware: 4× NVIDIA H200, tensor parallel 4. vLLM 0.31.0, without speculative decoding.

Thinking is on at reasoning_effort=xhigh, the chat template's own default, with the base model's recommended sampling — temperature=1.0, top_p=0.95, top_k=20, min_p=0.0 — used unchanged for Ours.

The generation cap is 253,952 tokens for most benchmarks. AA-LCR uses 131,072 and τ²-bench 65,536 per turn, because their documents and multi-turn transcripts occupy the rest of the window.

Benchmarks

Six benchmarks, each on its complete set: AIME 2026 (30 problems), HMMT Nov 2025 (30), LiveCodeBench v6 (175), AA-LCR (100), RealWorldQA (765) and τ²-bench (278 tasks across airline, retail and telecom).

Seeds and intervals

Independent runs per benchmark: 32 seeds on AIME 2026; 16 seeds on HMMT (Nov 2025); 8 seeds on AA-LCR, RealWorldQA and τ²-bench; 4 seeds on LiveCodeBench v6. Accuracy cells show the 95% t-interval across seeds — how much the answer moves when only the sampling seed changes, at Qwen's recommended temperature of 1.0.

Usage

The bf16 weights take about 360 GB; we serve them on four H200s (141 GB each).

HuggingFace Transformers

Requires transformers 5.16 or newer, the first release with this architecture.

from transformers import AutoModelForImageTextToText, AutoProcessor
model = AutoModelForImageTextToText.from_pretrained("bottlecapai/ThinkingCap-Qwen3.8-Flash-Next", dtype="bfloat16", device_map="auto")
proc = AutoProcessor.from_pretrained("bottlecapai/ThinkingCap-Qwen3.8-Flash-Next")

Check https://huggingface.co/Qwen/Qwen3.8-Flash-Next for recommended usage, sampling params etc.

vLLM

Requires vLLM 0.31.0 or newer. The reasoning parser returns the thinking in a separate reasoning field instead of inline in content before </think>, and the tool-call parser turns the model's XML tool calls into structured tool_calls — the same flags the base model's serving recipes use.

vllm serve bottlecapai/ThinkingCap-Qwen3.8-Flash-Next --tensor-parallel-size 4 --max-model-len 262144 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml

On H200 nodes without NVSwitch, our tensor-parallel servers crashed with illegal memory access when split across NVLink islands with vLLM's default FlashInfer all-reduce. We serve on one island with VLLM_ALLREDUCE_USE_FLASHINFER=0.

Speculative decoding

The checkpoint keeps the base model's MTP head. With 3 draft tokens and probabilistic drafting, vLLM 0.31.0 on four H200s accepted on average 2.77 tokens per decoding step with this model (the base model: 2.72), over AIME 2026 and RealWorldQA. vLLM issue #55357, MTP draft acceptance dropping to zero with repetition inside the thinking on this architecture, is still open; it did not reproduce in our test.

vllm serve bottlecapai/ThinkingCap-Qwen3.8-Flash-Next --tensor-parallel-size 4 --max-model-len 262144 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3,"draft_sample_method":"probabilistic"}'

Querying the server

The server speaks the OpenAI Chat Completions API. One request covers text, images and the thinking-effort knob:

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
r = client.chat.completions.create(
    model="bottlecapai/ThinkingCap-Qwen3.8-Flash-Next",
    messages=[{"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}},
        {"type": "text", "text": "What is happening in this picture?"},
    ]}],
    temperature=1.0, top_p=0.95,
    extra_body={"top_k": 20,
                "chat_template_kwargs": {"reasoning_effort": "xhigh"}},   # xhigh (default) | medium | low
)
print(r.choices[0].message.reasoning)               # the thinking
print(r.choices[0].message.content)                 # the answer

A text-only request is the same call with a plain string as content.

Quantized versions

Same checkpoint, chat template and license, quantized for smaller footprints:

• FP8 — bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-FP8 — vLLM; Qwen's own FP8 layout (FP8 routed experts and n-gram table), ≈186 GB
• NVFP4 (W4A16) — bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-NVFP4 — vLLM; 4-bit weight-only routed experts, ≈132 GB
• GGUF — bottlecapai/ThinkingCap-Qwen3.8-Flash-Next-GGUF — llama.cpp; BF16, Q8_0, Q6_K, Q4_K_M and IQ4_XS, with the vision projector

Where to find us

Need even more efficiency? Our enterprise versions go further — fewer thinking tokens still, tuned to your workload, at matched accuracy on your own tasks. Built for AI labs, inference providers and enterprises running models at scale. Deployed on your infrastructure, or in the cloud and region you choose. Talk to our team

License

Released under the base model's own license, the Qwen Community License 1.0 (full text in LICENSE). Its conditions carry over to this model, including the separate license from Qwen that Model-as-a-Service and AI work-assistant businesses need for commercial use.

Citation

If you use this model, please cite:

@misc{ThinkingCap-Qwen3.8-Flash-Next,
  title     = {bottlecapai/ThinkingCap-Qwen3.8-Flash-Next},
  author    = {Osusky, Adam and Lindauer, Jan and Jirkovsky, Adam and Mihal, Filip and Platek, Ondrej and Herel, David and Ihnatchenko, Luka and Bartek, Vojtech and Jirak, Jiri and Kubista, Daniel and Krus, Frantisek and Mikolov, Tomas},
  year      = {2026},
}