Grug 67B-A2B Antidoom RLVR1 step 12, agentic SFT on the thinking mix (step 465)
Research artifact. Evaluated only on the agentic benchmarks below. Not validated for production use.
open-athena/Grug-67B-A2B-Antidoom-RLVR1-Step12-2026.10.02 (Grug 67B-A2B 09.21 after RL) fine-tuned on marianna13/grug-sft-mix-v4-40M-think: a 40M-token mix of AgentTrove terminus-2 agent trajectories (teacher reasoning kept in Grug's think channel), synthetic recovery from Grug's own malformed-action styles, context-summarization handoffs and a small Nemotron instruction / math / science slice. This is the checkpoint at step 465 (about 2 epochs).
Evaluation
terminus-2 agent in harbor (Apptainer sandboxes), thinking on, 3× task timeouts, vLLM 0.26 on 4×A100, 32k input / 8k output tokens per call, no reasoning parser, temperature 1.0. Three independent runs per benchmark; solved tasks, mean (runs).
| benchmark | Antidoom RLVR1 step 12 (base) | this model |
|---|---|---|
| SWE-bench Verified, random 100 | 23.3 (23 · 24 · 23) | 41.7 (42 · 42 · 41) |
| Terminal-Bench 2.1, 74 validated tasks | 5.7 (6 · 7 · 4) | 10.0 (14 · 10 · 6) |
| TBLite 100 | 21.7 (22 · 21 · 22) | 25.3 (24 · 27 · 25) |
| terminus turns accepted (SWE-bench) | 85% | 97% |
| model format errors, share of turns (SWE-bench) | 8.0% | 1.2% |
For comparison, the same SFT applied to Grug 09.21 (without the RL stage, step 300, 4 runs) reaches 37.2 on SWE-bench, 10.3 / 74 on Terminal-Bench 2.1 and 23.0 on TBLite.
Known failure modes: most failed trials end with the agent declaring the task complete while it is still wrong (on SWE-bench about 12% of trials stop right after a test run that reported failures), and about a third of the Terminal-Bench failures are agent timeouts. The remaining format errors are mostly JSON escaping inside long keystrokes.
Usage notes
- The model always opens a
<|start_think|>block. The chat template defaults toReasoning: /think. - Serve with Grug's reasoning parser, or strip the think block before parsing the agent's action: some reasoning
contains
{, and a JSON parser that starts at the first brace rejects an otherwise valid action. config.jsonhas nolayer_types: attention is full on every 4th layer and the last layer, sliding-window (2048) elsewhere. The vLLM Grug plugin derives this pattern; check that your runtime does too.- Tokenizer and chat template are identical to the base model's.
Training
SkyRL main_sft with the Megatron backend on MareNostrum 5, starting from the base HF checkpoint, loss on assistant
turns (per-message train_loss from the dataset), Grug's own chat template with Reasoning: /think. Same recipe as
the 09.21 step-300 run (batch 32, 233 steps per epoch, learning rate 1e-5 with cosine decay); step 465 is about two
epochs.