Glistening-Gem-31B-v2.0

A Creative Merge  ·  31B  ·  Apache 2.0

⚠️ Heads up: this release has a known problem with its lm_head weights that affects the raw transformers weights and any quants built from it. It does not affect all quants already in circulation, but I recommend skipping this model in favor of sophosympatheia/Glistening-Gem-31B-v2.1. See Known Issues below before requantizing or relying on the transformers weights directly.

This is a merge of TheDrummer/Artemis-31B-v1, zerofata/G4-MeroMero-v2-31B, densenet/Gemma-4-31B-StyleTune-heretic-ara, and llmfan46/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic.

This merge improves upon sophosympatheia/Glistening-Gem-31B-v1.0 in several ways. v2 is more stable than v1 and is arguably more creative.

Known Issues

⚠️ Failed experiment: the fine-tuned lm_head does not merge well

This model's base, Gemma-4-31B-StyleTune-heretic-ara, carries a version of Gryphe's lm_head that was fine-tuned in isolation, with the rest of the model frozen, before this merge ever existed. My della recipe tried to protect and stabilize the tail layers around it (see the Merge Details / recipe file), but after actually testing the model with that real, isolated lm_head active, the result is noticeably worse and often incoherent. I'm calling this what it is: a failed experiment. I'm keeping the weights up as-is (rather than re-merging or rolling back) so this card can serve as an accurate record of what happened, but I don't recommend relying on the genuine lm_head for actual use.

Background: this repo briefly shipped with a broken lm_head, not a missing one

The transformers/safetensors weights in this repo originally shipped without a working lm_head.weight tensor at all (also missing: generation_config.json and a vision-embedding tensor), due to a bug in my mergekit fork's Gemma 4 architecture support that silently dropped those tensors during the merge. That bug has since been fixed, and this repo's weights have been patched to include the real, complete tensor set - so as of now, this repo's transformers weights are a fully accurate, complete reflection of the merge recipe, lm_head included. The problem described above is not a leftover artifact of that bug - it's what happens when the recipe's own lm_head is actually used correctly.

What this means for GGUF quants

llama.cpp's Gemma conversion falls back to reusing the token embedding matrix as the output layer whenever a separate lm_head.weight tensor isn't found in the source model. Because of the bug above, every GGUF quant made from this repo before this fix landed (before commit 12ceea82bb63f34c82ecab0626c51173e3991d6e on Fri, 14 Aug 2026 03:40:10 GMT) was built that way - using the tied token embeddings as a stand-in output layer, not the real, isolated-fine-tuned lm_head described above. Quants made after that commit are quantizing a different model altogether, one that performs much worse.

• If you already have a GGUF of this model that you've been happily using, keep using it. It's almost certainly one of the tied-embedding versions that works fine and is not affected by the lm_head issue. No action needed.
• If you (re)quantize this model now, be aware that a fresh conversion will correctly pick up the real, isolated lm_head and reproduce the degraded behavior described above. This is an unusual case where the "more technically correct" conversion is the one you don't want.
• If you use the transformers/safetensors weights directly, you are getting the real lm_head and will run into the same issues.

Net takeaway: some of the quants already circulating for this model, born from the original bug, are in this rare case the recommended way to run it. I'm leaving this note here so nobody mistakes a future "corrected" requantization for an improvement. This model has been superseded by sophosympatheia/Glistening-Gem-31B-v2.1, which fixes the problems and performs considerably better.

Sampler Tips

You can use the master import JSON in this repo (Glistening-Gem-v2-ST-Import.json) to deploy the sampler settings below. I recommend using these values as a starting point for your own experiments. It's not like the model falls apart if you deviate from these settings, but they should be a reliable starting point for most creative tasks.

A note on DRY + Adaptive-P: Some people recommend not mixing these, but in practice I haven't noticed any issues. Feel free to drop DRY entirely if you want. Adaptive-P is already designed to avoid repetitions.

Sampler Settings

Try these settings as a starting point.

Temp 0.8 Min-P 0.1 Adaptive-P Target 0.6 Adaptive-P Decay 0.75 DRY Mult. 0.8 DRY Base 1.8 DRY Allowed Length 6

Prompting Tips

You can download the Glistening-Gem-v2-ST-Import.json file from this repo and import it directly into SillyTavern to get system prompt, chat template, and sampler settings all in one go.

Donations

If you feel like saying thanks with a donation, I'm on Ko-Fi

Quantizations

Please see the sidebar of the model card where a link to quantizations can be found, or click here for the list of them.

License

Apache 2.0, inherited down from Gemma.

Merge Details

This is a merge of pre-trained language models created using mergekit.

Merge Method

This model was merged using the DELLA merge method using densenet/Gemma-4-31B-StyleTune-heretic-ara as a base.

See the model intro for more details and the mergekit_config.yaml file in the repo for the full recipe.