Qwythos-9B-Claude-Mythos-5-1M — SHQ8 (Selective Hybrid Quants)

Note on upstream v2: After these quants were made, the upstream release was updated to v2 with tokenizer normalization, embedded chat template improvements, and MTP variants. These quants are based on the original v1 source — they work fine with current llama.cpp for inference, but if you need the updated chat template or MTP support, grab the latest GGUFs from empero-ai/Qwythos-9B-Claude-Mythos-5-1M. MTP quants now available: wepiqx/Qwythos-9B-Claude-Mythos-5-1M-MTP-SHQ8-GGUF — same SHQ8 method with built-in MTP draft head for speculative decoding.

Note: File names contain Q5_K_M for HF parser compatibility only. These are not pure Q5_K_M — they're selective hybrid quants using Q8_0, Q6_K, IQ4_XS, Q5_K, and F16 across different tensor types. See each section for the exact per-tensor map.

Selective hybrid quantizations for Empero's Qwythos-9B-Claude-Mythos-5-1M — a full-parameter reasoning fine-tune of Qwen3.5-9B with 1M context and vision support.

Uses the exact same SHQ8 method and formulas as Qwable-9B-Claude-Fable-5-SHQ8-GGUF — same architecture, same imatrix, same quantization strategies. This is not a new SHQ9 series; it's the proven SHQ8 approach applied to a different fine-tune.

Status: Complete

Key finding: OptA formula transfers perfectly from Qwable (beats Q6_K PPL at 10% smaller). v2 shows a larger gap on Qwythos (+0.067) than on Qwable (+0.017) — IQ4_XS tensors impose a slightly higher cost on this fine-tune.

SHQ8-v2 vs stock quants: Despite the Q5_K_M name tag, SHQ8-v2 is actually better than a pure Q5_K_M — Q8_0 on critical attention paths + Q6_K on mid layers more than compensates for IQ4_XS on low-importance tensors. A stock Q5_K_M would be ~5.3 GB with PPL ~7.9 on Qwythos; v2 is 7.65 at nearly the same size. And vs Q3_K_M (~4.1 GB, PPL ~8.5+): v2 is only 1.6 GB larger but dramatically better quality. If you need the smallest possible file, consider v2 before dropping to low-bit quants.

Architecture

Identical to Qwable-9B-Claude-Fable-5 — same Qwen3.5-9B backbone:

Tensors, shapes, and importance distribution are identical to Qwable. All SHQ8 techniques transfer directly.

Imatrix

Reused Qwable-9B-Claude-Fable-5.imatrix.gguf — same architecture, same tensor layout, same importance distribution. Confirmed working — OptA formula produces identical size (6,303 MiB) on Qwythos.

SHQ8-OptA (Quality Champion)

Q5_K_M base + Q8_0 on attn_gate/qkv/ssm_alpha/beta all layers + blk.31 ffn_down Q8_0 + F16 norms + imatrix per-row.

~/llm-tools/llama.cpp/build/bin/llama-quantize \
  --imatrix /mnt/everything/qwen/source/Qwable-9B-Claude-Fable-5.imatrix.gguf \
  --output-tensor-type Q5_K \
  --token-embedding-type Q4_K \
  --tensor-type "output_norm.*=Q8_0" \
  --tensor-type "blk\.\d+\.attn_gate=Q8_0" \
  --tensor-type "blk\.\d+\.attn_qkv=Q8_0" \
  --tensor-type "blk\.\d+\.ssm_alpha=Q8_0" \
  --tensor-type "blk\.\d+\.ssm_beta=Q8_0" \
  --tensor-type "blk\.31\.ffn_down=Q8_0" \
  --tensor-type ".*attn_q_norm.*=Q8_0" \
  --tensor-type ".*attn_k_norm.*=Q8_0" \
  --tensor-type ".*ssm_conv1d.*=Q8_0" \
  --tensor-type ".*ssm_dt.*=F16" \
  --tensor-type ".*ssm_a=F16" \
  --tensor-type ".*attn_norm.*=F16" \
  --tensor-type ".*post_attention_norm.*=F16" \
  --tensor-type ".*ssm_norm.*=F16" \
  /mnt/everything/qwen/source/Qwythos-9B-Claude-Mythos-5-1M-BF16.gguf \
  /mnt/everything/qwen/output/Qwythos-9B-Claude-Mythos-5-1M-SHQ8-OptA.gguf \
  Q5_K_M

• PPL 7.4831 — beats Q6_K (7.5876) by −0.105 PPL at 10% smaller size
• Size 6,303 MiB vs Q6_K 7,008 MiB (10% smaller, 705 MiB saved)
• Same formula, same size, same quality transfer from Qwable

SHQ8-v2 (Compact Champion)

Q5_K_M base + tiered precision:

• Q8_0 — blk.0 (L0) + blk.26-31 attn_gate/qkv/ssm_alpha/beta, blk.31 ffn_down/attn_output, norms
• Q6_K — blk.1-25 attn_gate/qkv/ssm_alpha/beta
• IQ4_XS — ssm_out, attn_output, ffn_down (low importance)
• F16 — ssm_dt, ssm_a, all norms

Config: configs/SHQ8_v2.sh (same as Qwable SHQ8-v2, adapted for Qwythos paths)

• PPL 7.6542 vs Q6_K 7.5876 (+0.067, within 2× SE)
• Size 5,726 MiB vs Q6_K 7,008 MiB (18% smaller, 1,282 MiB saved)
• Larger gap on Qwythos than Qwable (+0.067 vs +0.017) — IQ4_XS slightly more costly here

Speed

v2 is ~8% faster thanks to fewer Q8_0 tensors reducing VRAM bandwidth pressure on Pascal's limited PCIe bus.

Coding Examples

Both quantizations generate full, working HTML/CSS/JS websites in a single pass at temperature 0.6 with the prompt:

"I'm a dev, my audience is youth. I like a creative/tech style. Write the full website code. This HTML will be our foundation."

SHQ8-OptA — mythos-SHQ8_temp-0.6.html

A complete dev portfolio in 1072 lines:

• Full hero, skills, projects, about, contact, and footer sections
• Animated grid background, floating elements, glow buttons
• Mobile menu, smooth scroll, active nav highlight, form with validation
• Stagger entrance animations for cards
• External deps: Google Fonts, Font Awesome

SHQ8-v2 — mythos-SHQ8-v2_temp-0.6.html

Same prompt, more streamlined output in 580 lines:

• Hero, projects, skills, and footer — focused layout
• Cleaner CSS, fewer sections, parallax effect
• No Font Awesome — lightweight approach
• External deps: Google Fonts only

At temp 0.6, both quants produce professional code. OptA goes for completeness (more sections, more features), v2 for conciseness. The reasoning fine-tune shows its quality — no repetition, no broken markup at this temperature.

Usage

Recommended sampling: Start with temperature 0.6, top_k 20, top_p 0.95, min_p 0. If you encounter looping or over-thinking, set repeat_penalty to 1.05 — this solves both issues without touching temperature. Be cautious with high temperatures — this is a reasoning fine-tune and can get unstable above 1.2.

Personal note from wepiqx: I've found that top_p 1.0 + min_p 0.05 often produces noticeably better results than top_p 0.95 + min_p 0. The wider top_p pool with a minimum probability threshold gives the model more creative freedom while still filtering out low-probability garbage. Give it a try.

llama.cpp

llama-cli \
  -m M-SHQ8-OptA-Q5_K_M.gguf \
  -p "Your prompt here" \
  -ngl 99 \
  --flash-attn on \
  -c 4096 \
  --temp 0.6 \
  --top-k 20 \
  --top-p 0.95 \
  --min-p 0

For server mode (recommended):

llama-server \
  -m M-SHQ8-OptA-Q5_K_M.gguf \
  -c 65536 \
  -fit off \
  -ngl 99 \
  --jinja --chat-template-file /path/to/chat_template.jinja \
  --flash-attn on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --port 8080 \
  --mmap \
  --temp 0.6 \
  --top-p 0.95 \
  --min-p 0 \
  --top-k 20 \
  --seed -1 \
  --parallel 1

VRAM usage: ~7.4 GiB / 8 GiB at -c 65536 with flash-attn + q8_0 KV cache. The Gated DeltaNet architecture is memory-efficient even at long context.

From wepiqx: This fine-tune can over-engineer at times, but overall it's a solid model. I'd love to see an MTP variant released — with MTP I could make an even better quantization with specialized draft model support for faster inference.

Ollama

1. Create a Modelfile:

FROM ./M-SHQ8-OptA-Q5_K_M.gguf

TEMPLATE """{{ .System }}
{{ .Prompt }}"""

PARAMETER num_ctx 8192
PARAMETER temperature 0.6
PARAMETER top_k 20
PARAMETER top_p 0.95
PARAMETER min_p 0

2. Build and run:

ollama create qwythos-SHQ8 -f Modelfile
ollama run qwythos-SHQ8

If the model loops or over-thinks, set repeat_penalty to 1.05 — this solves both issues.

LM Studio

1. Open LM Studio
2. Drag M-SHQ8-OptA-Q5_K_M.gguf into the app
3. Set GPU Offload to 99 layers
4. Enable flash-attention
5. Set context length to 8192+
6. Set sampling: temperature 0.6, top_k 20, top_p 0.95, min_p 0, repeat_penalty 1.0 (off)
7. If the model loops or over-thinks, set repeat_penalty to 1.05 — this solves both issues.
8. Start chatting

⚠️ Crucial Security & Safety Note (Uncensored Nature)

Please be aware that Qwythos-9B inherits a deeply uncensored base and is fine-tuned to engage substantively with high-stakes technical domains like offensive cybersecurity, red-teaming methodologies, clinical medicine, and advanced pharmacology without refusals, hedging, or generic disclaimers.

• For Users/Developers: This model does not surface safety boilerplate. It is critical to verify any specific identifiers, source code, or clinical data before execution or practical application.
• For Deployments: If you are using these SHQ8 quants in user-facing production applications, it is highly recommended to implement your own application-level moderation, review pipelines, or safety alignment layers depending on your target audience.

Files

Key Takeaways

1. OptA formula transfers perfectly — same architecture + same imatrix = same optimal quantization. PPL improved over Q6_K as expected.
2. Qwythos baseline is higher than Qwable (7.5876 vs 7.4394 at Q6_K) — 500M reasoning tokens and different eval domain (Wikitext-2).
3. v2 has a wider gap on Qwythos (+0.067 vs +0.017 on Qwable) — IQ4_XS tensors impose ~0.05 PPL extra cost on this fine-tune.
4. OptA remains the recommendation — best PPL at comfortable 6.3 GB. v2 only if 5.7 GB is required.

References

• Qwythos-9B-Claude-Mythos-5-1M
• Qwable-9B-Claude-Fable-5-SHQ8-GGUF — full methodology, importance analysis, PPL results (applies to Qwythos 1:1)