Gemma 4 26B-A4B — Decision-Point LoRA (l05 / l07)

A small LoRA for Gemma 4 26B-A4B IT and its fine-tunes. It makes swipes feel less like copies of each other.

What it does

• Swipes start differently. The opening words and the first sentence change from swipe to swipe. Paragraph openings vary too.
• More replies open with dialogue instead of the usual "The atmosphere in the room shifted…" scene description.
• Fewer stock phrases that the model repeats from reply to reply.
• Formatting stays intact. *actions* and "speech" are not broken.

What it does not do:

• It does not change what happens. The same story beat is delivered in a different way.
• Vocabulary and paragraph rhythm stay the same.
• With thinking / reasoning enabled, the effect is small: the reasoning already plans the opening.

Versions

Usage

• Apply on top of google/gemma-4-26B-A4B-it or its fine-tunes and merges.
• Strength: 1.0. LoRA r16, alpha 32. In KoboldCpp, scale 1.0 is the trained strength.
• Sampler: temperature ~0.85, min_p 0.05. The training target was built for this temperature.
• Files:
  • PEFT adapter: adapter_config.json, adapter_model.safetensors
  • GGUF LoRA (bf16) for llama.cpp / KoboldCpp: <file name>.gguf

How it was made

The idea. An aligned model makes its most "templated" choices at a few decision points. The first words of a reply decide its whole shape, and the IT model picks almost the same ones every time. The base model is much more varied at these points.

The teacher. For every response token, the target distribution is:

• first 3 words of the reply and of each paragraph (up to 8 tokens): a mix of IT and base, base share λ;
• words 4–5: base share λ·2/3, then λ·1/3 — a soft handoff back to IT;
• after a sentence end: the base votes only on "start a new paragraph or continue";
• everywhere else: the IT distribution itself.

Guards.

• Only IT decides punctuation, word continuations and the end of the turn.
• In *asterisk* scenes, the base can open a paragraph only with * or a quote.

Sampler compensation. The mix is min-p filtered (0.02) and raised to the power 0.85. Sampling at temperature 0.85 then reproduces the mix itself.

Distillation.

• Loss: KL to the teacher at decision points; KL to IT (weight 5) everywhere else.
• Data: 200 roleplay contexts, 400 responses (IT generations and mixed IT+base generations).
• LoRA: r16, alpha 32, layers 13–23, attention (q/k/v/o) and the dense MLP. LR 5e-5, 2 epochs.

The result needs only one model at inference. The base model is used only during training.

Related work

• BACo — inference-time routing between a base and an aligned model (Wang et al., ICML 2026). This adapter distills a narrow, decision-point version of the same idea.
• CreativeInstruct — distills BACo-style generations into a LoRA at the level of whole responses (Sahu et al., 2026).
• Switch Generation — base, SFT and aligned checkpoints take turns writing a response (Feng et al., ICLR 2026).