Gemma-4-26B-A4B agentic LoRA — run2 (OpenHands format)

A small LoRA for google/gemma-4-26B-A4B-it that nudges it toward agentic coding: multi-turn tool use and edit → run → fix loops. Applied at low strength with thinking off, it keeps the model's score on an agentic coding benchmark and finishes in fewer output tokens.

Sibling: run4 — same data, rewritten into OpenCode's tool vocabulary. run2 measured better of the two, even when driven by OpenCode.

Files

Use it with llama.cpp — at strength 0.25, thinking off

llama-server -m gemma-4-26B-A4B-it-Q8_0.gguf \
  --lora-scaled gemma4-agentic-run2-tok5M-lora-bf16.gguf:0.25 \
  --jinja --chat-template-file chat_template.jinja --reasoning off \
  --ctx-size 131072 --parallel 1 -ngl 999 \
  --temp 1.0 --top-p 0.95 --top-k 64

• Strength matters more than anything else. Use --lora-scaled …:0.25, not --lora (which applies 1.0). Above ~0.3 it starts costing score: 39/42 at 0.5. 0.15–0.25 is the useful range.
• Thinking off. It was trained and measured with thinking disabled; with thinking on it measured the same as stock Gemma (no benefit, no harm).
• chat_template.jinja is the one shipped in the base instruct repo. Any GGUF of the instruct model works; the numbers below are on Q8_0.

With transformers/PEFT: PeftModel.from_pretrained(base, "Myric/gemma4-26b-a4b-agentic-lora-run2", subfolder="tok5M"), then scale the adapter (or merge at 0.25). The config's base_model_name_or_path is a local path — pass the base model yourself.

Measured

42-task agentic coding suite (a real OpenCode tool loop, graded by hidden pytest), thinking off, DGX Spark:

Same score, fewer tokens: run2 @ 0.25 used fewer output tokens than stock on 31 of 42 tasks across its three runs (sign test p ≈ 0.002). On a typical task the saving is modest; the large totals gap also includes stock stalling on individual hard tasks. Token counts vary a lot between identical runs, so read these as a direction, not a ratio.

Training

• Adapted: attention q/k/v/o_proj and each layer's dense MLP gate/up/down_proj; LoRA rank 32, alpha 64, no dropout (37.2 M parameters). The 128 routed experts are frozen.
• Data: ~5 M tokens (95 steps) of a mix: 45% agentic — resolved nebius/SWE-rebench-openhands-trajectories, windowed to ~10k tokens in their original OpenHands format; 35% general chat and 20% tool calls from HuggingFaceTB/smoltalk + Team-ACE/ToolACE, with refusals filtered out.
• Loss only on tokens the model generates, rendered with the instruct repo's own chat template and tool tokens.
• Trained on one DGX Spark (GB10).

License

Use is subject to the Gemma Terms of Use. Training data: SWE-rebench OpenHands trajectories (CC-BY-4.0, Nebius), SmolTalk (Apache-2.0, Hugging Face), ToolACE (Apache-2.0).