Loading…
Loading…
StyleTune for your QAT model — without training anything.
Google's QAT base + Gryphe's StyleTune voice. Best of both. One 1.5 GB download. Zero extra models.
This voice is cast from Gryphe/Gemma-4-26B-A4B-StyleTune-V2 and matched to Google's QAT checkpoint — google/gemma-4-26B-A4B-it-qat-q4_0-unquantized. If your GGUF is built from the QAT weights, this is the variant that fits it exactly.
Two independent optimizations, now combined:
Google's QAT trains the model to survive quantization. Per Google, QAT gives 54% less perplexity drop at Q4_0 versus standard post-training quantization (Google Developers Blog) — 5,000 steps of fake-quantized forward passes with KL divergence to the BF16 teacher. Your Q4_0 GGUF keeps noticeably more of the original model's fidelity.
StyleTune then changes how it writes: per Gryphe's benchmarks — 200 roleplay prompts, greedy 0.0 — 52% fewer clichés (1.141 → 0.551 per 100 words) and only 19.9% shared trigram vocabulary vs the base instruct.
The old tradeoff was: QAT model or StyleTune model, pick one. Now you don't pick. If you already have a QAT GGUF, this voice turns it into a QAT-fidelity, StyleTune-voiced model — and you download 1.5 GB instead of 50+ GB.
# 1. Get the voice tool (one-time): https://huggingface.co/Wiself/voice
python3 voice.py path
# 2. Cast onto your QAT-derived GGUF
voice cast ./gemma-4-26b-a4b-it-qat-q4_0-Q4_0.gguf voice.safetensors --out ./voiced/gemma-4-26b-a4b-qat-styletune.gguf
Append --speak to force the voice through output.weight (invented if the target lacks it and geometry survives; GGUF only).
Run it:
llama serve -m ./voiced/gemma-4-26b-a4b-qat-styletune.gguf --jinja
No QAT finetune was needed. No extra model was downloaded. Your model, smarter voice, less quantization loss.
The QAT checkpoint's lm_head weights differ slightly from the standard instruct's (that's the point of QAT — the weights learned to live with quantization). This voice is cast as a delta against the normal google/gemma-4-26B-A4B-it (voice − base where base is the standard instruct, not QAT). When you later do delta + QAT_head → Q8_0, you add StyleTune's style to the QAT head — correct math, matched foundation.
| Your GGUF is built from | Use |
|---|---|
google/gemma-4-26B-A4B-it (standard) |
V2 voice |
google/gemma-4-26B-A4B-it-qat-q4_0-unquantized |
this voice |
The most popular QAT-adjacent models are abliterated/uncensored variants — for example llmfan46/gemma-4-26B-A4B-it-qat-q4_0-unquantized-uncensored-heretic. We tested this exact pairing:
voice delta → voice cast with the delta) — clean output, no loops, StyleTune phrasing intact.On abliterated or heavily modified targets, start with the delta path (base is the normal instruct — correct math):
voice delta voice.safetensors --base google/gemma-4-26B-A4B-it
voice cast ./model.gguf delta-voice.safetensors --out ./voiced/model.gguf
Why it works: a delta carries only the difference from the base head, so the shift to MoE routing is minimal. Tiny nudges can flip which expert fires next in MoE models; the delta keeps the nudge small while keeping the style. Full story in the Voice tool card.
voice.safetensors — the style delta (voice − base), F16, shape [262144, 2816], ~1.5 GBvoice.json — metadata: source, dtype, shape, base| Target | Works? |
|---|---|
| QAT-derived Gemma 4 26B A4B GGUFs (any quant) | ✅ primary target |
| Abliterated QAT variants (heretic et al.) | ✅ via the delta path above |
| Standard (non-QAT) 26B A4B GGUFs | ✅ works, but the V2 voice is the exact match |
| Gemma 4 other sizes / non-Gemma | ❌ shape mismatch |
voice info voice.safetensors → [262144, 2816] · F16 (delta marker present).google/gemma-4-26B-A4B-it-qat-q4_0-unquantized, Gemma terms.google/gemma-4-26B-A4B-it-qat-q4_0-unquantized