Qwen3.8-27B ByteShape IQ4_XS-3.84bpw — ASCII-P1M vocab
ByteShape's Qwen3.8-27B IQ4_XS-3.84bpw (ShapeLearn per-tensor quantization) with its vocabulary pruned to ASCII plus math/typography symbols, using bsaleh03/ASCII-Condensed-prune-tools (--policy P1M).
English and code only. See Limitations.
| File | Qwen3.8-27B-ByteShape-IQ4_XS-3.84bpw-ASCII-P1M.gguf |
| Size | 12.25 GB (source 13.08 GB) |
| Vocab | 129,272 tokens (source 248,320) |
| sha256 | d533c568ba028fb7bc24d646d9f0259341a388fb2535e559fd0bc62275a55a4d |
| Source sha256 | 89434f23dc89c5f990894e3fe9fdad19d88c370f0d3638a176f29933f218b78b |
| Architecture | qwen35 (hybrid Gated DeltaNet + attention), 65 blocks, MTP head kept |
| Parameters | Qwen3.8-27B (27.3B). The Hub shows 26.1B because it counts tensor shapes, and the pruned vocab removes 119,048 rows from both token_embd and output (~1.22B parameters). Every transformer weight is unchanged. |
What changed
Only token_embd.weight (IQ4_XS, 0.675 → 0.352 GB) and output.weight (Q6_K, 1.043 → 0.543 GB) were touched. Rows are gathered in quantized space: no dequantize/requantize, so every kept row is bit-identical to the source. The tokenizer arrays and merges were rewritten and the special-token ids remapped. The 256 byte-fallback tokens, all specials and all partial-UTF-8 fragments are kept, so any text can still be tokenized.
verify_prune.py --policy P1M passed every check:
- all 864 non-vocab tensors byte-identical to the source
- sampled vocab rows identical to their source rows
- metadata preserved
- kept set exactly what P1M specifies
Tool commit: 376a426d5c6b30f28e03c6efaa9edcf85048773f.
Measurements
Measured on an RTX 5060 Ti 16 GB with llama.cpp v0.4.1.
Perplexity: ctx 4096, 30 chunks, f16 KV. Lower is better, and the figures are deterministic, so they compare exactly across rows.
| model | code (mixed TS/JS project source) | code, ASCII-only lines | wikitext-2 |
|---|---|---|---|
| ByteShape IQ4_XS-3.84bpw (source) | 1.6671 | 1.6490 | 5.9480 |
| this file | 1.7156 | 1.6488 | 5.9612 |
| Unsloth UD-IQ4_XS (reference) | 1.6583 | — | 5.8322 |
On ASCII-only text the prune costs nothing (1.6488 vs 1.6490). The +2.9% on the mixed code corpus comes entirely from its 1.27% non-ASCII characters, mostly Bangla string literals.
Context and speed: whole model on GPU, q4_0 KV cache, -ub 512, MTP off.
| ctx | VRAM allocated | prefill | decode |
|---|---|---|---|
| 96k | 13,952 MiB | 939 t/s | 28.7 t/s |
| 128k | 14,391 MiB | 939 t/s | 28.8 t/s |
| 160k | 14,873 MiB | 895 t/s | 27.7 t/s |
A 16 GB card with a light desktop fits about 128k of context; 160k needs a nearly idle desktop.
Agentic coding eval (10 executable tasks, thinking on, temp 1.0 / top_p 0.95 / top_k 20): 8/10, 18,381 tokens total. One of the two failures is a grading artifact: the answer ended with a usage-example code block, which the grader took for the solution.
Tool calling and mid-conversation system messages work with a patched chat template (see below).
Usage
llama-server -m Qwen3.8-27B-ByteShape-IQ4_XS-3.84bpw-ASCII-P1M.gguf \
-c 131072 -ngl 99 -fa on --cache-type-k q4_0 --cache-type-v q4_0 \
--jinja --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0
- MTP: the head is still in the file; add
--spec-type draft-mtp --spec-draft-n-max 2if you have the VRAM for the draft context. - Claude Code and similar agents: the stock Qwen3.8 template raises
System message must be at the beginning.on mid-conversation system messages. Patch that branch to render them as normal system blocks.
Limitations
- Non-Latin text breaks. Scripts outside ASCII and the P1M symbol set (Bangla, CJK, Arabic, Cyrillic, …) fall back to one token per UTF-8 byte. The model was never trained on those byte sequences, so it reads them as garbled text. A Bangla prompt got a romanized reply saying the input looked garbled; the full-vocab model answers it correctly. For multilingual use, take the source file instead.
- Accented Latin (é, ü, ñ) is not in P1M either, so it also degrades.
- Not affiliated with ByteShape or the Qwen team. All numbers above come from one machine.
Credits
- Base model: Qwen/Qwen3.8-27B (Apache-2.0)
- Quantization: ByteShape (ShapeLearn)
- Vocab pruning: bsaleh03/ASCII-Condensed-prune-tools