Callsign "Casanova": SubMaroon and Indexnusrefather present to you:
Kitchoon-v3.1-26B-A4B
The new LoRA was trained on a very diverse dataset and baked into the existing Kitchoon V3. My goal was to diversify Gemma's template responses, and I may have achieved this goal.
I tried to get Gemma to produce slightly more varied responses. I managed to break her formulaic reply patterns a bit. A characters now interact with each other more, and the responses no longer follow the same tired, repetitive structure.
[!CAUTION] ⚠️EXPERIMENTAL MODEL⚠️
It may be unstable, unfortunately. This is an experimental release.
WARNING: Thinking doesn't work properly!
The Kitchoon series is designed as an RP- and ERP-oriented model, focusing on character development and featuring a neutral writing style.
What was changed
| Kitchoon-V3 | Kitchoon-V3.1 | |
|---|---|---|
Attention OV (v_proj + o_proj) |
stock | LoRA, layers 10–29, 36 targets, baked at 0.5 |
Dense MLP (gate_proj + up_proj + down_proj) |
stock | LoRA, layers 10–20, 33 targets, baked at 0.5 |
lm_head, QK, MoE experts, router, embeddings, vision tower |
stock | identical |
☕ Support
If this model was useful to you, you can support my work with a small donation.
It helps me pay for GPU time and keep releasing new models and datasets.
Gram (TON network): UQB6cmEHWIcuiwsjgO_C0Uz4fyX3Puj09BAB3hEVVpDDcQ7K
Boosty donation: Link
Details
- OV and dense MLP were trained together as one LoRA, but baked with separate strengths (both at 0.5).
- MoE experts and the router were never touched — checked by module name and by matrix shape.
- Baked with stochastic bf16 rounding, so the baked strength matches the tested one.
| Parameter | Value |
|---|---|
| Rank / alpha | OV 32 / 64, MLP 16 / 32 |
| Learning rate | 2.5e-5, cosine |
| Epochs | 1 (754 steps) |
| Context | 8192 tokens |
| Eval loss | 4.32 → 1.41 |
The dataset is sloppier than the base, so training on it naively would bring slop back. The model still sees these spans in context — it just gets no gradient for reproducing them:
- Slop phrases. Stock cliché phrases plus ~400 n-grams that showed up across many different characters in the dataset ("voice dropping to a", "a flicker of", "tilts her head"...).
- Slop constructions. "Not just X, but Y", "It wasn't an X; it was a Y" — the antislop base doesn't catch these.
- "X — or Y?" endings. The final either-or question sentence of a reply.
- Echoing the user. A first sentence that repeats the user's words back.
- Card echo. Text copied verbatim from the character card.
In testing, slop stayed at the level of the clean base.
I would like to express my immense gratitude to Indexnusrefather, with whom I work, for providing the dataset.
DatasetEnglish RP dataset, ~2.8k multi-turn conversations built on character cards.
Cleaned before training:
- removed cards with underage characters and real people;
- fixed ~640 system prompts that instructed the model to write for the user;
- long conversations split into 8192-token chunks instead of being truncated;
- the first and/or last paragraph randomly trimmed in part of the replies, to break up identical openings and hook-question endings;
- character name prefixes (
Name:) stripped from replies.
Using it
[!Note] Recommended settings:
Parameter Value Temperature 0.85Min-P 0.05Repetition penalty / DRY off Thinking off
Credits
Thanks to Tetratype for the antislop base, to coder3101 for the heretic version it is built on, and to Sam Paech for the auto-antislop pipeline. Thanks to the entire 26B-Suite team for their intellectual support.
Special thanks to Naphula and redaihf. You guys are awesome!
And special thanks, of course, to Indexnusrefather for his excellent dataset.