Soyuz-4B GRPO — Q4_K_M GGUFs
llama.cpp Q4_K_M quants of the GRPO LoRA fine-tunes of Soyuz-4B, where the reward is a
separate Claude Code instance grading each rollout. Each version is a merged LoRA → GGUF.
| file | version | what it adds | measured effect |
|---|---|---|---|
v1.Q4_K_M.gguf |
v1 | GRPO on general-coding tasks, free-form quality rubric | flat learning (+0.004, noisy judge) |
v2.Q4_K_M.gguf |
v2 | + criteria-checklist reward + library/codebase-fix tasks | clean gain (+0.045, 19/34 up); greedy base→GRPO +0.06 |
v3.Q4_K_M.gguf |
v3 | + creative tasks + per-turn recovery reward (harsh grounded critique) | raw +0.057, turn-0 +0.071 (recovery internalised into first-try); ⚠️ regresses on agentic terminal-bench (general-coding GRPO doesn't transfer to shell agents) |
v5.Q4_K_M.gguf |
v5 | + single-shot tasks + Hermes-agent tool-calling tasks | best run: ALL +0.081, turn-0 +0.101; single-shot +0.208, hermes turn-0 +0.109 (learns the right tool / clarify / approval on the first try) |
Serve
llama-server -m v3.Q4_K_M.gguf --host 0.0.0.0 --port 8080 -ngl 99 --jinja --ctx-size 32768
Best for single-response coding / writing / rewriting. For multi-step shell agents prefer the
base Soyuz-4B. Suppress <think> for one-shot app/code generation (the model over-plans otherwise).
Details
Full GRPO recipe + honest effect analysis (incl. the negative agentic-transfer result) on the v3 model card: AlexWortega/qwen35-4b-soyuz-grpo-v3. W&B: https://wandb.ai/alexwortega/fast-qwen35-sft