Caphlon-Model
A 6.87 GiB GGUF build of Qwen 3.8 27B — full 27B reasoning quality, laptop-sized footprint.
Built and benchmarked on 25 October 2026. No distillation, no student model: the 27B topology survives. Footprint was cut by removing representation redundancy — vocabulary merge-table reduction and hybrid-interval-aware layer pruning — on top of an Unsloth Dynamic IQ2_S base.
Model details
| Base model | Qwen 3.8 27B (Apache-2.0) |
| Intermediate | Unsloth Dynamic IQ2_S |
| Architecture | 60-layer hybrid: gated-delta-net (majority) + full-attention (minority) |
| File | Caphlon-Model-IQ2_S.gguf — 6.87 GiB (7,375,122,272 bytes) |
| SHA-256 | 8487b4e443834659b230e0bd529674ad65e068fc00380b996aaf11c873d63bc0 |
| Format | GGUF (llama.cpp) |
| Context | up to 32K tested |
| Language | English |
KV cache lives only on the full-attention layers; gated-delta-net layers carry fixed-size recurrent state, so memory stays small and context scales cheaply.
Changes vs. the base quant
| Step | Change | Saved |
|---|---|---|
| 1 | Vocabulary merge table reduced to a 100k target (closure-preserving selection) | −649 MiB |
| 2 | Layers 12–15 removed (4 layers, aligned to the hybrid interval) | −302 MiB |
| 3 | Weights kept at IQ2_S — no further quantization damage | — |
Benchmarks
Measured 2026-10-25, Tesla T4 (16 GB), llama.cpp, temperature 0, thinking disabled.
| Test | Caphlon-Model | Base quant | Result |
|---|---|---|---|
English perplexity (wiki.test.raw, 512 ctx × 40 chunks) |
6.6088 | 6.3638 | +3.85% |
| HumanEval, 30 problems, pass@1 | 29/30 | 30/30 | 96.7% |
| MMLU, 502-question subset | 344/502 | 370/502 | 92.9% retained |
File-edit throughput (--spec-type ngram-mod) |
29.7 tok/s | 12.7 tok/s | 2.34× |
| Free-form prose throughput | 12.4 tok/s | 12.7 tok/s | — |
Hardware requirements
| Setup | Minimum | Comfortable |
|---|---|---|
| GPU, full offload | 8 GB VRAM (8K ctx) | 12 GB VRAM (32K ctx) |
| CPU only | 8 GB RAM (mmap) | 16 GB RAM |
| Apple Silicon | 16 GB unified | 24 GB+ |
| Disk | 7 GB | — |
Usage
llama-server -m Caphlon-Model-IQ2_S.gguf \
-ngl 99 -c 32768 -np 1 \
-b 4096 -ub 1024 --flash-attn on \
--spec-type ngram-mod --spec-ngram-mod-n-match 24 \
--spec-ngram-mod-n-min 48 --spec-ngram-mod-n-max 64 \
--jinja --port 8080
Send chat_template_kwargs: {"enable_thinking": false} on chat requests, otherwise
the model spends the token budget on internal reasoning.
Optional, only when context-bound: --cache-type-k q8_0 --cache-type-v q4_0
(measured gain −152 MB, but greedy output is then no longer byte-identical).
License and attribution
Weights are a derivative of Qwen 3.8 27B (Apache-2.0); the upstream license governs them and permits commercial use with notices retained. Documentation, naming, and evaluation scripts by the author are Apache-2.0.
"Caphlon" is a project mark. Apache-2.0 grants no trademark rights: you may redistribute the weights, but you may not present a modified build as official, endorsed, or authored by this project.
Citation
@misc{caphlon2026,
title = {Caphlon-Model: A 6.87 GiB lightweight build of Qwen 3.8 27B},
author = {demiralpdev},
year = {2026},
month = {10},
note = {GGUF, IQ2\_S base, vocabulary-reduced and layer-pruned},
url = {https://github.com/demiralpdev/Caphlon-Model}
}
- GitHub: https://github.com/demiralpdev/Caphlon-Model
- Project page: https://demiralpdev.github.io/Caphlon-Model/