SVP-V-GRPO · COCONUT GPT-2
A COCONUT GPT-2 (124M)
latent-reasoning model post-trained with SVP-V-GRPO — reinforcement learning whose
rollout exploration comes from weight-space perturbation of the attention value
projections. For each layer and rollout, SVP independently samples Gaussian noise and
rescales coefficients in the fixed SVD basis of W_V as sᵢ → sᵢ(1+αgᵢ).
Only the value-projection weights (the V columns of each c_attn) differ from the base
checkpoint. The perturbation is a training-time exploration mechanism and is not used
at deployment: inference is ordinary greedy decoding.
This repository contains the paper's GPT-2 SVP-V-GRPO checkpoint from epoch 9 B=32, lr=6e-5 run.
Results
Clean greedy decoding, max_new_tokens=64, exact-match on the final answer.
| Model (GPT-2 124M) | GSM8K | GSM-Hard | MultiArith | SVAMP | ASDiv-A | GSM-Plus |
|---|---|---|---|---|---|---|
| COCONUT (base) | 34.1 | 7.7 | 80.9 | 35.6 | 60.2 | 17.4 |
| SLPO | 34.9 | 7.6 | 82.8 | 34.3 | 58.7 | 18.2 |
| SIM-CoT | 44.7 | 9.3 | 90.5 | 40.6 | 67.2 | 21.5 |
| CoDi | 42.5 | 9.3 | 91.9 | 40.0 | 65.4 | 23.1 |
This model (SVP on W_V, α=0.6) |
50.3 | 11.3 | 93.4 | 43.6 | 71.2 | 27.6 |
Usage
This is a fixed-length continuous-thought model: the prompt ends with
<|start-latent|>, six latent steps feed each step's last hidden state back as the next
input embedding, <|end-latent|> closes the latent phase, and the answer is then decoded
as ordinary tokens.
from transformers import AutoTokenizer, GPT2LMHeadModel
REPO = "." # path to this repository
tok = AutoTokenizer.from_pretrained(REPO)
model = GPT2LMHeadModel.from_pretrained(REPO).eval().cuda()
# then run the two-phase latent loop: 6 latent steps, then decode the answer
A reference implementation of the loop ships with the SVP code release; the original COCONUT inference code works as well.
Training
| Base | ModalityDance/latent-tts-coconut (COCONUT GPT-2, 124M) |
| Data | 383,891 eligible prompts from GSM8K-Aug |
| Algorithm | GRPO: G=32 rollouts per prompt; 2B=64 candidate prompts per iteration with up to B=32 mixed-outcome groups retained; μ=2 optimization passes; centered advantages; symmetric clipping ε=0.2; k3 KL (β=0.02) to the frozen base |
| Exploration | SVP with α=0.6: independent Gaussian coefficient perturbations for each Value-projection layer and rollout, held fixed through prefill and all six latent steps; clean answer decoding; SVD factors refreshed every 50 iterations |
| Trained parameters | V columns of every c_attn (≈ 7M of 124M) |
| Optimizer | AdamW, constant lr=6e-5, seed 0, no warmup or weight decay, gradient clipping at 1.0; epoch-9 checkpoint from 15 epochs |
| Reward | Final-answer correctness only |
License
MIT, inherited from the base checkpoint (which derives from openai-community/gpt2).