Gemma-4 Vain 26B-A4B (GGUF)

Vain is a roleplay and creative-writing finetune of Gemma 4 26B-A4B (MoE, ~4B active parameters). It aims for a distinct, natural prose voice instead of the usual stock RP phrasing, while keeping the speed of the A4B MoE.

Built on an uncensored (heretic) base, so it follows your lead rather than refusing. Use responsibly.

Files

Because it's an MoE with only ~4B active parameters, it also runs acceptably with part of the experts offloaded to CPU if a quant doesn't fit your card.

All quants were made from a full-precision (F16) master with an importance matrix. The calibration text was a standard calibration set plus roleplay transcripts.

Quant quality, measured against the full-strength model on held-out roleplay text:

Q3_K_M was tested and left out. It lost noticeably more quality than IQ4_XS for only ~0.6 GB of savings.

Recommended settings

Tested in KoboldCpp + SillyTavern with the model's built-in Gemma 4 chat template.

DRY plus temperature 0.7 was a clear improvement in blind tests: 31 of 48 head-to-head wins over the defaults, with better logic and less repetition.

Reply length: Vain handles short, medium and long replies. Steer it with a line in your system prompt or card, for example "Reply length: one short paragraph" or "Reply length: several paragraphs."

One tip: don't add "keep track of everything" style instructions to the card. In testing, that made it fixate and repeat questions.

How it was made

• Method: inspired by the Sprinkle approach. A LoRA (rank 32, alpha 32, on all attention and MLP projections) was trained as plain text completion on the pretrained Gemma 4 26B-A4B base. It was then merged at full strength into trohrbaugh/gemma-4-26B-A4B-it-heretic-ara-v2, an uncensored version of the instruct model.
• Training data: about 17.6M tokens of curated prose, fiction and roleplay text, trained for one epoch.
• Training hardware: two consumer GPUs (RTX 5080 + RTX 5070 Ti), about 6.6 hours.

Evaluation

Vain was blind-tested in 8 multi-turn roleplay scenes, 12 or more turns each. The scenes were built to test logic and continuity: item custody, counts and money, darkness and blindness, audio-only characters, and character rules such as vows of silence or a robot that never says "I".

Six outside judges (frontier LLMs, anonymized outputs, rotated order) ranked it against other Gemma 4 26B-A4B roleplay finetunes and our own variants. Vain placed 2nd overall on average, behind Gryphe's StyleTune-V2 and ahead of the other community finetunes tested.

Strengths judges noted: voice and characterization, in-character warmth, and following card rules.

Known weaknesses:

• It still makes about 2 continuity slips per 12-turn scene, such as an item ending up in the wrong hands or a running count drifting.
• It sometimes assumes the user's gender when the card doesn't state it.
• Like most RP models, it has a few favourite phrases. DRY helps.

License

Apache 2.0, the same as Gemma 4 and the heretic base.

Credits

• Google DeepMind for Gemma 4
• trohrbaugh for the heretic-ara-v2 base, and Heretic for the uncensoring method
• Peregrine, whose Sprinkle finetunes inspired the training approach
• llama.cpp for GGUF conversion and quantization