Please use Kitchoon-V3!
Kitchoon-v2-26B-A4B
Kitchoon-v1 OV LoRA and head training, ported onto my base StyleTune-QK-Heretic.
Thinking is now working properly. But the model is more creative in the non-thinking mode! The slope primarily comes from StyleTune but is slightly smoothed out by a head injection from Kitchoon-v1.
The Kitchoon series is designed as an RP- and ERP-oriented model, focusing on character development and featuring a neutral writing style.
What was changed
| Base (StyleTune-QK-Heretic) | Kitchoon-V2 (OV and lm_head from Kitchoon-V1) |
|
|---|---|---|
Attention OV (v_proj + o_proj) |
stock | LoRA, layers 10–29, 36 targets, baked at 0.45 |
lm_head |
stock | trained separately, blended with the stock head at 0.40 |
| QK, MoE experts, router, MLP, embeddings, vision tower | stock | identical |
☕ Support
If this model was useful to you, you can support my work with a small donation.
It helps me pay for GPU time and keep releasing new models and datasets.
Gram (TON network): UQB6cmEHWIcuiwsjgO_C0Uz4fyX3Puj09BAB3hEVVpDDcQ7K
Boosty donation: Link
Details
- OV: Kitchoon's OV LoRA (layers 10–29) at 0.6.
- Head: Kitchoon's head training added to the StyleTune head, Pantheon excluded:
head = head_StyleTune + 0.6*(head_Kitchoon - head_Pantheon).
| Parameter | Value |
|---|---|
| Rank / alpha | 32 / 64 |
| Learning rate | 2e-5, cosine |
| Epochs | 1 (931 steps) |
| Context | 3328 tokens |
| Eval loss | 4.67 → 1.45 |
The head was trained as a full tensor with the body frozen.
- Slop was masked out of the loss. Stock cliché phrases plus ~400 n-grams that showed up across many different characters in the dataset ("voice dropping to a", "a flicker of", "tilts her head"...) got no gradient. Same for text copied verbatim from the character card.
- Rare tokens were frozen. Rows for tokens seen fewer than 5 times in the targets stayed bit-identical to the base.
I would like to express my immense gratitude to Indexnusrefather, with whom I work, for providing the dataset.
DatasetEnglish RP dataset, ~2.8k multi-turn conversations built on character cards.
Cleaned before training:
- removed cards with underage characters and real people;
- fixed ~640 system prompts that instructed the model to write for the user;
- long conversations split into chunks instead of being truncated;
- the first and/or last paragraph randomly trimmed in part of the replies, to break up identical openings and hook-question endings;
- character name prefixes (
Name:) stripped from replies, since Pantheon was trained without them.
Using it
[!Note] Recommended settings:
Parameter Value Temperature 1.0Min-P 0.05Thinking off — the training data had no reasoning blocks, and thinking mode was not tested.
What to expect
In testing, compared to the base:
- characters stick closer to the card — they pull in side characters, habits and lore from the description on their own;
- more dialogue, more humor, less of the character explaining themselves;
- scenes carry forward from turn to turn instead of resetting.
Known quirks, still there:
- characters sometimes repeat the user's question back before answering;
- replies like to end on an "X — or Y?" question;
- smirk and the em dash have not fully left the building.
Feedback
This is an experiment, and I mostly test it on my own cards, so my coverage is narrow. If you try it, I would really like to know:
- How it holds up in long sessions — scene logic, who is where, who is talking to whom.
- Whether the quirks above bother you in actual play.
- Your sampler settings, if you got clearly better or worse results than with the defaults.
Open a thread in the Community tab, I read all of them.
Credits
Thanks to Gryphe for Pantheon-Reasoning, and to Vortex5 for the heretic version it is built on. Thanks to the entire 26B-Suite team for their intellectual support.
Special thanks to Naphula and redaihf. You guys are awesome!
And special thanks, of course, to Indexnusrefather for his excellent dataset.