This is a standalone BF16 model derived from google/gemma-4-26B-A4B-it. One model supports effort 10, 25, 50, 75, 100, and default through the prompt.
Benchmark results at effort 10, with completed coverage, are reported below.
Place this exact line first in the system prompt, changing 10 to another supported setting:
Reasoning Effort: 10 (range 1–100; higher values request more thorough reasoning)
Append any other system instructions after it on a new line. Keep thinking enabled
with enable_thinking=True. For default, omit the effort line while keeping
thinking enabled. These are reasoning preferences, not hard token limits
or percentages of an inference token budget. Other effort values have not been validated.
effort_presets.json and examples/ include all five effort settings and default.
Use Transformers with Gemma 4 support. BF16 weight files total approximately 52 GB, with additional memory needed for inference.
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
model_id = "krystofmitka/ThinkingNuke-gemma-4-26B-A4B"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
model_id, dtype=torch.bfloat16, device_map="auto"
)
messages = [
{"role": "system", "content": "Reasoning Effort: 10 (range 1–100; higher values request more thorough reasoning)"},
{"role": "user", "content": "Solve 3x + 7 = 28. Explain your answer."},
]
inputs = processor.apply_chat_template(
messages, add_generation_prompt=True, enable_thinking=True,
tokenize=True, return_dict=True, return_tensors="pt"
).to(model.device)
with torch.inference_mode():
output = model.generate(
**inputs, max_new_tokens=12288,
do_sample=True, temperature=1.0, top_p=0.95, top_k=64
)
print(processor.tokenizer.decode(
output[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=False
))
The decoded response includes thinking and final-answer channels. Adjust the output
limit for your application; reaching it can truncate reasoning or the final answer.
For vLLM, use a version supporting Gemma 4, serve this repository with
--served-model-name effort-model, and use the request bodies in examples/.
Base = original Gemma 4 26B-A4B, no effort instruction. Ours = ThinkingNuke, effort 10. Accuracy is mean ± the 95% t-interval half-width in percentage points; a single seed has no CI yet. Reduction = 100 × (1 − Ours / Base) in mean thinking tokens; “more tokens” indicates an increase. Only fully scored seeds count, and coverage is shown for both models. Pending and not-run entries have no measured comparison. Macro averages use all 12 benchmarks with equal weight.
Baseline results: 13/312 passes complete; evaluation running. Comparison updated 2026-11-18 16:09 Europe/Prague. The table refreshes on the next 30-minute check after a benchmark pass completes.
Accessible results and completed coverage| Benchmark | Accuracy / success (%) | 95% CI across seeds (%) | Mean thinking tokens | Questions per seed | Seeds completed / planned |
|---|---|---|---|---|---|
| GPQA-Diamond | 75.25 | 74.18–76.33 | 2,942 | 198 | 10/32 |
| MMLU-Pro | 83.84 | 83.76–83.93 | 1,910 | 12,032 | 4/4 |
| MMMLU | 82.84 | 82.50–83.18 | 452 | 9,996 | 4/4 |
| AIME 2026 | 79.63 | 74.76–84.50 | 8,155 | 30 | 9/64 |
| HMMT February 2026 | 64.31 | 60.32–68.31 | 15,844 | 33 | 9/64 |
| HMMT November 2025 | 72.22 | 70.00–74.44 | 9,763 | 30 | 9/64 |
| LiveCodeBench v6 | 73.84 | 72.82–74.87 | 6,802 | 175 | 9/32 |
| AA-LCR | 44.50 | 41.67–47.33 | 1,323 | 100 | 8/8 |
| LongBench v2 | 58.78 | 58.07–59.50 | 4,860 | 370 | 8/8 |
| RealWorldQA | 69.82 | 69.05–70.58 | 2,660 | 765 | 9/16 |
| IFBench | 75.67 | 74.69–76.64 | 2,887 | 300 | 9/16 |
| τ²-bench | 34.65 | 32.85–36.45 | 2,417 | 278 | 9/16 |
Only full, successfully scored seeds enter these aggregates; partial answers and failed passes are excluded. Mean scores and thinking lengths average completed seeds. All 12 benchmarks receive a first pass before additional repeats. Different rows may have different repeat counts.
Confidence intervals are 95% Student t intervals across completed sampling seeds, requiring at least two seeds. They measure generation variability on the fixed question selections, not uncertainty from sampling new questions. With only a few seeds they can be wide. Thinking-token lengths are means without confidence intervals.
Metric definitions. IFBench reports strict prompt-level pass rate. τ²-bench reports the unweighted mean success rate across its domains; its thinking-token count is summed over all evaluated-agent turns per conversation, excluding the user simulator. Other thinking-token counts are per answer and exclude final-answer tokens.
Evaluation setup. This model at effort 10, full BF16, thinking enabled, vLLM 0.33.0 / EvalScope 1.14.1, context 262,144, output cap 65,536, temperature 1.0, top-p 0.95, top-k 64. Fixed selections are counted above; long-context items that cannot reserve the full answer budget are excluded. The frozen original google/gemma-4-E2B-it is the local AA-LCR judge and τ² user simulator, using its own tokenizer with no effort instruction. This is not an exact replication of BottleCap evaluation prompts or helper models. The original 26B comparison uses the baseline protocol documented below; other effort settings have not been evaluated here.
Evaluated weight revision: 0dc255ba0a474ba3bea757ac0db9b9660a2d2cc4. Helper revision: 4adf15748de9e1a0097b6274cee9c15836c41c37.
Machine-readable results · Evaluation protocol
AA-LCR judging recovery: the frozen E2B judge allowance was raised from 4,096 to 8,192 tokens during the third seed after two verdicts exhausted the original allowance. All evaluated-model answers were retained; the third seed’s judgments are recomputed through EvalScope’s review-rerun mode at the new allowance. Earlier completed seeds retain their original judgments. The evaluated model’s 65,536-token budget and the τ² user-simulator settings are unchanged.
Running on one RTX PRO 6000: 13/312 benchmark passes complete. Updated 2026-11-18 16:09 Europe/Prague; this section refreshes after each completed pass.
The original google/gemma-4-26B-A4B-it is evaluated with the same question selections (verified identical), sampling settings, limits, and scoring as ThinkingNuke above, with no effort line. ThinkingNuke numbers are its published effort-10 results. Every benchmark gets a first pass before any repeats.
Completed results appear in the Base and Reduction columns of the formatted benchmark table above.
Accessible comparison results and completed coverage| Benchmark | Original: accuracy (%) | Original: mean thinking tokens | ThinkingNuke: accuracy (%) | ThinkingNuke: mean thinking tokens | Thinking-token reduction (%) | Seeds (original · ThinkingNuke) |
|---|---|---|---|---|---|---|
| GPQA-Diamond | 80.05 ± 3.18 | 11,193 | 75.25 ± 1.08 | 2,942 | 73.7 | 2/32 · 10/32 |
| MMLU-Pro | 84.70 | 4,944 | 83.84 ± 0.08 | 1,910 | 61.4 | 1/4 · 4/4 |
| MMMLU | 85.10 | 2,256 | 82.84 ± 0.34 | 452 | 80.0 | 1/4 · 4/4 |
| AIME 2026 | 93.33 | 12,706 | 79.63 ± 4.87 | 8,155 | 35.8 | 1/64 · 9/64 |
| HMMT February 2026 | 69.70 | 26,462 | 64.31 ± 4.00 | 15,844 | 40.1 | 1/64 · 9/64 |
| HMMT November 2025 | 86.67 | 19,395 | 72.22 ± 2.22 | 9,763 | 49.7 | 1/64 · 9/64 |
| LiveCodeBench v6 | 77.14 | 12,153 | 73.84 ± 1.03 | 6,802 | 44.0 | 1/32 · 9/32 |
| AA-LCR | 66.34 ± 17.03 | 6,054 | 44.50 ± 2.83 | 1,323 | 78.1 | 2/8 · 8/8 |
| LongBench v2 | 61.35 | 10,102 | 58.78 ± 0.71 | 4,860 | 51.9 | 1/8 · 8/8 |
| RealWorldQA | 72.42 | 2,120 | 69.82 ± 0.76 | 2,660 | -25.5 | 1/16 · 9/16 |
| IFBench | 76.00 | 6,139 | 75.67 ± 0.98 | 2,887 | 53.0 | 1/16 · 9/16 |
| τ²-bench | not run | — | 34.65 ± 1.80 | 2,417 | — | 0/16 · 9/16 |
Accuracy is the mean over completed seeds, ± the half-width of the 95% t-interval across seeds when there are at least two. Seeds are sampling repeats of the same questions, so the two models can rest on different numbers of seeds; only fully scored seeds count. Thinking tokens are per answer and exclude the final answer.
Evaluated revision: 4d7ae4984b7db7de8f8457170b3f1a419ee76d52. Same vLLM 0.33.0 / EvalScope 1.14.1 setup, context 262,144, output cap 65,536, temperature 1.0, top-p 0.95, top-k 64. ThinkingNuke was evaluated mostly on H200 GPUs, the original model on a single RTX PRO 6000; hardware does not change the protocol.
Machine-readable results · Evaluation protocol
weights_manifest.json records the size and SHA-256 of each weight shard.
The upstream model was released by Google DeepMind under Apache 2.0. This repository
contains modified weights. See LICENSE, NOTICE, and
BASE_MODEL_CARD.md for upstream attribution and model information. Upstream
benchmark scores describe Google's original model.
model-00003-of-00006.safetensors 9.8 GBmodel-00004-of-00006.safetensors 9.8 GBmodel-00005-of-00006.safetensors 9.8 GBmodel-00002-of-00006.safetensors 9.8 GBmodel-00001-of-00006.safetensors 9.6 GBmodel-00006-of-00006.safetensors 2.8 GB