Stardom

LuneLight

Gemma4 31B 01 Overview

Wasn't really planning on doing another Gemma model, but was getting really sick of GLM and Mimo so decided to mess around with some smaller models again for a change of pace. This is a finetune on top of Scotoma 2, a finetune by Aes Sedai that got rid of a couple annoying writing tics Gemma 4 31B has.

It's pretty different in terms of approach from my other models. I use nothing but GRPO and DPO for this model, with no SFT at all. I'm pretty burnt on dataset wrangling and it was a fun experiment to try and come up with reward functions to guide Gemma towards writing better itself.

Just going by my personal evaluation suite, I think this model hit those goals. The intelligence hasn't taken a hit from Scotoma 2 which it was based on and I definitely prefer the writing.

Supports both thinking and non thinking, I personally prefer to use it with thinking enabled.

02 SillyTavern Settings Suggested Roleplay Format ActionsIn plaintext Dialogue"In quotes" Thoughts*In asterisks* Recommended Samplers Temp0.8 - 1.0 MinP0.05 Instruct Gemma 4 - Think Gemma 4 - NoThink 03 Quantizations GGUF iMatrix 04 Evaluation 05 Creation Process

Creation Process: GRPO > DPO

Stage 1 — GRPO.

Reward per reply sanity0 – 1× slop0 – 1× phrasing0.70 – 1× cadence0.75 – 1× echo0.80 – 1 sanityStrict Gate. Replies either passed the sanity check or failed and got scored 0. Looked for truncated replies (think loops etc.), the model escaping into a different language and other hacks. HTML, OOC notes, zalgo text and ASCII etc. were excluded so the model didn't get rewarded or punished for them. slopLooked for overused words and collocations of words. Rather than searching for ngrams, a sliding window was used to find words that appeared near each other too often. Allowed the model to use slop words if the phrasing was natural, but also allowed the reward to punish when it just varied the phrase or an adjective in the middle. Also negation variations were bundled with the slop reward. phrasingCredit for human phrases that the base model rarely does ("as soon as", "at least"). Basically the slop check but in reverse, things found in human data but not LLM writing. Naturally filters out spelling and grammar as it uses the same trick of looking for collocated words instead of ngrams. cadencePenalises monotone sentence openers ("He flinches. He looks down. He bites his lip."). Encourages the model to start sentences differently. echoPenalises parroting the user's last message back at them, measures multiple types of echoing and only punishes overused formats. In more detail

The five terms are multiplied, so the model is encourage to not over optimize for any one aspect of the reward. The highest rewards can only be met in a response that satisfies every reward.

Sanity

Out-of-character notes are removed along with stat blocks, message headers etc. For the length and language checks, HTML, markdown, zero-width characters and Zalgo marks are stripped and look-alike Cyrillic or Greek letters are mapped back to Latin.

A reply scores 0 if it:

• was truncated due to any reason
• has a missing, broken or leaked thinking block, or any thinking when thinking is off,
• is a short refusal or stub, or is abnormally short,
• switches into another script or has too few common English words, beyond what the prompt itself uses.

With thinking on, the first 600 thinking words are free. After that the score falls smoothly, reaching 0 at 1,200 words.

Slop

Tense normalisation. To avoid *most* of the problems with comparing different types of AI and human written text, it's normalized where possible ("drops" and "dropped" both become "drop").

Mining the dictionary. Every pair of content words that appear within five words of each other in the model's replies is summed. A pair has to appear across at least 5% of scenarios and then pass two tests:

• Dunning's log-likelihood test (Dunning 1993), which checks whether the two words show up together more often than chance. This is a pass/fail filter.
• Delta P (Gries 2013), which measures how much seeing one word raises the chance of seeing the other and take the stronger of the two directions. It filters out pairs of common words that only meet because both are frequent, and it sets the base penalty for each pair that is kept.

The dictionary ends up with four kinds of entries:

• Hub words: words where at least half of all uses sit inside one of their overused pairs or frames, along with those partner pairs. These are template phrases that the model has learned to vary and easily avoid detection via ngrams (they just swap to the second most common option).
• Standalone pairs: strongly linked pairs that are widespread on their own.
• Frames: a hub word joined to the grammar word next to it ("scent of", "lets out"). Penalising content pairs alone lets the model keep the habit and swap the noun ("scent of cedar", "scent of rain"), and frame allows that to be punished. A frame only counts if the model uses it at least 7× as often as humans, so ordinary grammar doesn't get caught.
• Excess words: single words used at least 8× as often as in human writing, across at least 20% of scenarios ("shimmering", "ozone", "predatory"). Traditional slop words the model grossly overuses.

Comparing against human writing. Everything above is weighted by how much more the model uses it than humans do. Anything the model uses at twice the human rate or less is dropped, which leaves normal phrases, templates and words like "eyes closed" or "head shakes" alone. Above that, the penalty grows with the gap.

Scoring a reply. Each reply is scanned for pairs, frames, stacked adjectives ("a thick, suffocating cloud") and excess words. A longer cliché like "heart hammering against ribs" counts once instead of once for every pair inside it. When a reply hits the same hub word several times, the strongest hit counts in full and each extra hit counts for less, using BM25 saturation (Robertson & Walker 1994). Importantly, because the reward looks primarily at pairs / frames / hubs, the model is not punished when it uses a slop word in a new way (i.e. The scent was strong is allowed vs the scent of ozone which gets punished). The total is measured per 1,000 words.

Negation. This catches "not X, but Y" emphasis: a negation such as "not", "never" or "n't" followed within 12 words by a contrast such as "but", "rather", "instead", a dash or semicolon, or a next sentence starting with "Instead". Guards skip interrupted speech and asides. The patterns the model uses 12× to 82× more than humans add to the slop score.

Calibration. Phrase and negation hits are added into one slop density and a single curve is applied. Up to the human 75th percentile, slop scores 1. Above it the score decays exponentially, at a rate set so a typical base-model reply scores 0.4 while the densest human writing still scores at least 0.8.

Phrasing

A bonus for everyday expressions like "at least", "one of" and "as soon as" that humans use and the model avoids. Just looks for 2 and 3 word phrases that are widespread across forum RP threads and sources, also common in other human writing, and used at least 2.5× more by humans than by the model. Phrases with past tense verbs, personal pronouns or scene-specific nouns were removed so the list works in any scene and tense, and overlapping phrases were merged.

Each phrase counts once per reply and is weighted by how rarely the model being trained uses it, so phrases it learns give rewards can't be abused. Credit is measured per 1,000 words and reaches full marks at the human 25th percentile. A reply with none scores 0.70, and replies under 40 words are left at 1.

Cadence

Dialogue is removed and the first word of each narration sentence is taken as its opener. One leading word like "Then" or "Slowly," is skipped first so it can't hide a "He… He… He…" chain. Two measures are taken:

• The normalised Herfindahl–Hirschman index, equivalent to Simpson's index for a finite sample (Simpson 1949), for how concentrated openers are across the whole reply. It is 0 when every opener differs and 1 when they are all the same.
• I also manually count how often an opener comes back in the next sentence or the one after, since a long reply can cheat the index.

Both are free up to the human 75th percentile and reach the full penalty at the base model's 90th percentile. The worse of the two sets the score, down to 0.75. Replies with fewer than 5 narration sentences get the human average of 0.96, so dropping narration isn't a way around the check.

Echo

Same idea as negation, regex checks whether the reply's dialogue or short italic thoughts copy the user's last message. Before matching, stutters and repeated letters are collapsed, contractions are expanded and "I" and "you" are treated as the same, so "You're crazy" echoed as "I'm crazy?" still counts. Character names, titles, lines the user asked the character to say and replies like "I love you too" are ignored.

Then echoing is split into a few types depending on what check caught it. Models tend to only overuse a few types of echoing, so only those types are punished

Stage 2 — DPO.
After GRPO I liked the model, but I noticed it had found a few gaps in my reward suite. "as soon as" got pushed a bit too successfully, going from barely used to x43 human rate and "leaning" becoming overused. I also wanted the model to be more aware of the scene so I got Gemma to write a hint for itself of was anyone watching / nearby and is there anything in the way. Two vague questions that the answers was then placed into the system prompt during generation and removed after and rejected pairs then generated without the hint. Even doing only 30 steps of DPO, the model appears to have learnt the idea well and was notably more spatially aware than before and the two phrases it had found were reduced.