ThinkingCap: Nemotron 3.5 Lightning 30B-A3B

ThinkingCap brings our shorter-thinking post-training to NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 (NVIDIA, 2026), keeping its performance on challenging and agentic tasks while delivering a substantial reduction in thinking verbosity. ThinkingCap Nemotron 3.5 Lightning 30B-A3B cuts reasoning tokens by 23% on average (12% to 31% depending on the benchmark) and holds an average accuracy of 71.4% against the base model's 72.2%. Check our blogpost for more details.

A better ThinkingCap Nemotron is ready for enterprise today. This open checkpoint is our public release; the enterprise version goes further, with fewer thinking tokens still at matched accuracy, and we tune it to your workload. Deployed on your infrastructure, or in the cloud and region you choose. Contact us if you're interested.

ThinkingCap vs. the base model

Video comparison of the original model and the ThinkingCap version, prompted with the same input. Both models run on the same hardware and use the same inference settings.

Shorter thinking, more finished work

On a busy server handling many concurrent requests, fewer thinking tokens pay off twice. Each task finishes sooner, and each running sequence uses less KV cache, so fewer requests get evicted and more can run side by side.

Visualisation of an example scenario where both models receive an unlimited stream of requests, showing how many tasks each can complete within a given time window. The ThinkingCap version uses fewer reasoning tokens, resulting in a smaller KV cache footprint and allowing more tasks to run in parallel.

Both models run on identical setups and get the same chemistry problem, with 20 minutes to solve it. ThinkingCap uses 23% fewer thinking tokens and completes 25 tasks in that time, compared to 11 for the base model. We explain why in Halve the thinking trace length, quadruple the work.

Token efficiency and benchmark performance

Evaluation details

Models

Base nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 against bottlecapai/ThinkingCap-NVIDIA-Nemotron-3.5-Lightning-30B-A3B, shown as Ours.

Metrics

• Accuracy (Base / Ours) — fraction of correct answers. The τ²-bench score is an unweighted mean over its three domains (airline, retail, telecom), not pooled over tasks.

• Thinking tokens (Base / Ours) — mean length of the <think> trace, answer excluded. τ²-bench is a multi-turn agentic episode, so for that row the count is the reasoning summed over every turn of the episode, not a single trace.

• (Thinking token) Reduction — the relative change between the two mean columns beside it, (Ours − Base) / Base, where each mean is taken over every question and seed of the benchmark. A saving is shown as a green ↓ percentage.

• Macro average (bottom row) — equal-weight mean across benchmarks of each column, including the reduction: it is the mean of the per-benchmark reductions, not the ratio of the two token figures beside it.

We separately track two trace-quality failure modes. Truncation: on the single-turn benchmarks, the <think> trace never closes because the model hits the generation cap while still reasoning, so no answer is produced. On τ²-bench it is an episode-level flag, so it compounds over turns: an episode is marked if any of its turns hits the per-turn cap. Looping: the model repeating the same reasoning until it never finishes, detected with a compression-ratio test on each trace (on τ²-bench, on the episode's reasoning joined across turns). Both stay below 1% overall and both improve: truncation 0.43% → 0.22% and looping 0.30% → 0.16%, equal-weight across benchmarks.

Serving

Hardware: NVIDIA RTX PRO 6000 Blackwell. vLLM 0.27.1 with an FP8 KV cache and no speculative decoding, except SciCode, which both models ran with MTP (num_speculative_tokens=3).

Thinking is on (enable_thinking=True, the chat template's own default), with the base model's recommended sampling — temperature=1.0, top_p=0.95 — used unchanged for Ours.

The generation cap is 253,952 tokens for most benchmarks. AA-LCR and τ²-bench use 65,536, because their documents and multi-turn transcripts occupy the rest of the window.

Benchmarks

Ten run the complete set: AIME 2026 (30 problems), HMMT Feb 2026 (33), HMMT Nov 2025 (30), GPQA-Diamond (198), IFBench (300), AA-LCR (100), τ²-bench (278 tasks across airline, retail and telecom), LiveCodeBench v6 (175), SciCode (288 subproblems — the test split) and MMLU-Pro (12,032 — the whole test split).

MMMLU is the one subset: a 10,000-question random sample of the multilingual set, drawn with a fixed seed so every condition sees the same questions.

Seeds and intervals

Independent runs per benchmark: 64 seeds on AIME 2026, HMMT (Feb 2026) and HMMT (Nov 2025); 32 seeds on GPQA-Diamond; 16 seeds on LiveCodeBench v6, AA-LCR, IFBench and τ²-bench; 8 seeds on SciCode; a single seed on MMLU-Pro and MMMLU. The seed count decides what the accuracy interval means. Multi-seed rows show the 95% t-interval across seeds — how much the answer moves when only the sampling seed changes, at NVIDIA's recommended temperature of 1.0. MMLU-Pro and MMMLU run a single seed over 12,032 and 9,996 questions, so they instead show a 95% Wilson interval over the question outcomes — how much the answer would move on a different draw of questions. The two measure different sources of variance and should not be read against each other.

Usage

HuggingFace Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("bottlecapai/ThinkingCap-NVIDIA-Nemotron-3.5-Lightning-30B-A3B", dtype="bfloat16")
tok = AutoTokenizer.from_pretrained("bottlecapai/ThinkingCap-NVIDIA-Nemotron-3.5-Lightning-30B-A3B")

Check https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 for recommended usage, sampling params etc.

vLLM / SGLang

Serve the bf16 model with either engine. NVIDIA's serving recipes for the base model apply unchanged: vLLM 0.27.1, the version our evaluation ran, and SGLang from the lmsysorg/sglang:dev-nemotron3-5-lightning container. The commands below carry the base model's parsers. The reasoning parser returns the thinking in a separate reasoning / reasoning_content field instead of inline in content, and the tool-call parser turns the model's XML tool calls into structured tool_calls.

# vLLM
vllm serve bottlecapai/ThinkingCap-NVIDIA-Nemotron-3.5-Lightning-30B-A3B --async-scheduling \
  --reasoning-parser nemotron_v3 --enable-auto-tool-choice --tool-call-parser qwen3_coder

# vLLM on an 80 GB GPU (H100): NVIDIA's 1×H100 flags added
vllm serve bottlecapai/ThinkingCap-NVIDIA-Nemotron-3.5-Lightning-30B-A3B --async-scheduling \
  --max-num-seqs 128 --enable-prefix-caching --mamba-backend flashinfer --mamba-ssm-cache-dtype float16 \
  --enable-mamba-cache-stochastic-rounding --mamba-cache-philox-rounds 5 \
  --reasoning-parser nemotron_v3 --enable-auto-tool-choice --tool-call-parser qwen3_coder

# SGLang
sglang serve --model-path bottlecapai/ThinkingCap-NVIDIA-Nemotron-3.5-Lightning-30B-A3B \
  --reasoning-parser nemotron_3 --tool-call-parser qwen3_coder

The first vLLM command and the SGLang command run as shown on a B200, and the SGLang command also runs on an H100. On an 80 GB GPU the first vLLM command stops at startup (max_num_seqs (1024) exceeds available Mamba cache blocks), so use the H100 variant there. The base model card lists the recipes for other GPUs.

Speculative decoding

ThinkingCap keeps the base model's built-in MTP head and works with NVIDIA's two external drafters, DSpark and DFlash. With every drafter, on both engines, its acceptance length (the tokens produced per decoding step) matches the base model's (ThinkingCap / base model):

Measured on a B200 with one request at a time over six prompts, thinking on. Each drafter drafts the same number of tokens per step on both engines: 5 for MTP, as in the base model card's SGLang command, and 7 for DSpark and DFlash, the draft length NVIDIA benchmarks them at. On vLLM 0.27.1, DSpark accepts only about 3% of its drafted tokens with either checkpoint, which makes decoding slower than with no drafter; the same drafter works on SGLang.

To enable a drafter, append its flags to the commands above (tested on a single B200):

# SGLang + MTP
--speculative-algorithm EAGLE --speculative-draft-model-path bottlecapai/ThinkingCap-NVIDIA-Nemotron-3.5-Lightning-30B-A3B \
--speculative-num-steps 5 --speculative-eagle-topk 1 --speculative-num-draft-tokens 6
# SGLang + DSpark
--speculative-algorithm DSPARK --speculative-dspark-block-size 7 --speculative-num-draft-tokens 8 \
--speculative-draft-model-path nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark
# SGLang + DFlash (the block counts the verified token: 8 = 7 drafted; 4 on H100)
--speculative-algorithm DFLASH --speculative-dflash-block-size 8 \
--speculative-draft-model-path nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DFlash

# vLLM + MTP
--speculative-config '{"method":"mtp","num_speculative_tokens":5}'
# vLLM + DSpark
--speculative_config.model nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark \
--speculative_config.num_speculative_tokens 7
# vLLM + DFlash
--speculative_config.model nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DFlash \
--speculative_config.num_speculative_tokens 7

Querying the servers

Either server speaks the OpenAI Chat Completions API. Thinking is on by default and is switched with a chat-template argument:

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")   # SGLang: port 30000
r = client.chat.completions.create(
    model="bottlecapai/ThinkingCap-NVIDIA-Nemotron-3.5-Lightning-30B-A3B",
    messages=[{"role": "user", "content": "How many primes are there below 100?"}],
    temperature=1.0, top_p=0.95,
    extra_body={"chat_template_kwargs": {"enable_thinking": True}},   # False answers directly
)
print(r.choices[0].message.reasoning)               # the thinking (`reasoning_content` on SGLang)
print(r.choices[0].message.content)                 # the answer

Quantized versions

Same checkpoint, chat template and license, quantized for smaller footprints and faster serving:

• FP8 — bottlecapai/ThinkingCap-NVIDIA-Nemotron-3.5-Lightning-30B-A3B-FP8 — FP8 per-channel weights and per-token activations, vLLM, 35 GB; Hopper and Blackwell; pre-release
• GGUF — bottlecapai/ThinkingCap-NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF — Q4_K_M and f16 (25–66 GB), llama.cpp; tested on CUDA; pre-release

Where to find us

License

Released under the base model's own license, the OpenMDW License Agreement 1.1 (full text in LICENSE).

Citation

If you use this model, please cite:

@misc{ThinkingCap-NVIDIA-Nemotron-3.5-Lightning-30B-A3B,
  title     = {bottlecapai/ThinkingCap-NVIDIA-Nemotron-3.5-Lightning-30B-A3B},
  author    = {Osusky, Adam and Lindauer, Jan and Jirkovsky, Adam and Mihal, Filip and Platek, Ondrej and Herel, David and Ihnatchenko, Luka and Bartek, Vojtech and Jirak, Jiri and Kubista, Daniel and Krus, Frantisek and Mikolov, Tomas},
  year      = {2026},
}