Qwopus3.8-27B-Coder-Flash-TQ
Engine: tqLLM 1.2.3 (docker pull ytan2000/tqllm:1.2.3 or ghcr.io/turbo-tan/tqllm:1.2.3).
Jackrong/Qwopus3.8-27B-Coder-Flash, a coding fine-tune of Qwen3.8-27B, in TQ_L (14.63 GB) for the tqLLM engine, with a DFlash2 draft model for exact speculative decoding. Against Jackrong's own Q4_K_M (16.8 GB) on the same engine and GPU: 2.2 GB smaller, decode 24 % faster, equal or better on every quality row below.
Credits
| What is in these files | Credit |
|---|---|
| Qwopus3.8-27B-Coder-Flash fine-tune (BF16 source) | Jackrong (Jackrong/Qwopus3.8-27B-Coder-Flash) |
| Qwen3.8-27B base weights, MTP head | Qwen team, Alibaba (Qwen) |
| Output head and token embeddings (Q6_K / Q4_K), tokenizer, chat template | from Jackrong/Qwopus3.8-27B-Coder-Flash-GGUF Q4_K_M |
| DFlash2 draft model weights and method | Jian Chen, Yesheng Liang, Zhijian Liu, z-lab (arXiv 2602.06036) |
| Rotation-based quantisation, the lineage of the TQ4 format | Google Research, TurboQuant |
Credits for the engine (KV cache, speculative decoding, kernels): tqllm.com/credits.
Files
| File | Size | What it is |
|---|---|---|
Qwopus3.8-27B-Coder-Flash-TQ_L.gguf |
14.63 GB | the model: TQ4 body, Q6_K output head, Q4_K embeddings; 24 GB GPUs and GB10 |
drafters/Qwen3.8-27B-DFlash2-TQ3.gguf |
0.79 GB | DFlash2 draft model (Qwen3.8-27B), TQ3 weights |
mtp-vocab/atx_65536.txt |
0.4 MB | the draft model's token list (65,536 ids) |
drafters/NOTICE |
licence notice for the draft model |
hf download YTan2000/Qwopus3.8-27B-Coder-Flash-TQ Qwopus3.8-27B-Coder-Flash-TQ_L.gguf drafters/Qwen3.8-27B-DFlash2-TQ3.gguf mtp-vocab/atx_65536.txt --local-dir models
Run
This file needs tqLLM: its TQ4 tensors use tqLLM's own GGUF type (300), so stock llama.cpp, Ollama and LM Studio cannot load it.
docker run --rm --gpus all -p 127.0.0.1:8086:8086 -v "$PWD/models:/models:ro" \
-e MODEL_FILE=/models/Qwopus3.8-27B-Coder-Flash-TQ_L.gguf \
-e DRAFT_MODEL_FILE=/models/drafters/Qwen3.8-27B-DFlash2-TQ3.gguf \
-e DRAFT_VOCAB_FILE=/models/mtp-vocab/atx_65536.txt \
-e KV_FORMAT=tq3 -e CONTEXT=32768 -e REASONING=on -e MAX_TOKENS=4096 \
ytan2000/tqllm:1.2.3
The image runs on x86-64 (RTX 30 / 40 / 50) and arm64 (GB10 / DGX Spark). The server is OpenAI-compatible
(/v1/chat/completions). All options: docker run --rm --gpus all --entrypoint tqserve ytan2000/tqllm:1.2.3 --help.
- Drafter: the Qwen3.8-27B DFlash2 draft model proposes several tokens and the 27B checks them in one pass; every accepted token is exact, so the output is the model's own text, only faster. It was trained on the base model and still works well for this fine-tune (decode below).
- Long agent loops: in long tool-use sessions (75K+ tokens of history) this fine-tune can fall into a repeated
reasoning loop more often than the base Qwen3.8-27B; Jackrong's Q4_K_M behaves the same (it is the fine-tune, not the
quantisation). A presence penalty of 1.5 (
"presence_penalty": 1.5in the request) keeps it out of them.
Quality
Same engine, same GPU (RTX 3090), both files:
| TQ_L | Jackrong Q4_K_M | |
|---|---|---|
| Size | 14.63 GB | 16.8 GB |
| GSM8K (200, greedy) | 96.0 % | 96.0 % |
| MMLU-Pro (291-question subset, thinking off) | 83.8 % | 81.4 % |
| KLD vs the Qwopus BF16, prose + code (lower is closer) | 0.016 | 0.017 |
| KLD, prose / code | 0.019 / 0.012 | 0.016 / 0.017 |
| KLD, chat with thinking | 0.20 | 0.24 |
| Same top token as BF16 (all text) | 93.0 % | 93.2 % |
KLD: mean per-token KL divergence from the BF16 model on the same 1024-token chunks (6 prose, 5 code, 5 chat). Chat with thinking is far from BF16 for both files alike, which points at the fine-tune, not the quantisation. Over all 16 chunks: 0.075 vs 0.086, within noise (paired bootstrap). MMLU-Pro on 291 questions has a standard error of about 2.2 points.
BenchLoop by task (RTX 3090, same engine, both files):
| Task | TQ_L | Q4_K_M |
|---|---|---|
| coding | 93.8 | 93.8 |
| toolcall | 86.7 | 86.7 |
| dataextract | 88.1 | 87.4 |
| reasonmath | 80.0 | 80.0 |
| instructfollow | 87.8 | 87.8 |
Speed (RTX 3090 24 GB, tqLLM 1.4.0 release candidate, greedy, 64K context, q8 KV, 4 prompts x 2, 400 tokens each)
| TQ_L | Q4_K_M | |
|---|---|---|
| Decode tok/s, DFlash2 drafter | 196.4 | 158.9 |
| Decode tok/s, no drafter | 51.6 | 40.5 |
| Prefill tok/s, 14,401-token prompt | 1,822 | 1,700 |
| Prefill tok/s, 29,887-token prompt | 1,740 | 1,607 |
Decode speed with a drafter depends on the text: code and copy-heavy answers accept more drafts than open prose.
The run command above, as written, with ytan2000/tqllm:1.2.3 (32K context, tq3 KV, DFlash2 drafter):
| Hardware | Ready | 512-token answer | 26,199-token prompt: first token |
|---|---|---|---|
| RTX 3090 24 GB | 136 s | 149.2 tok/s | 14.3 s |
| GB10 (DGX Spark) | 18 s | 61.7 tok/s | 23.7 s |
Format
TQ4 (4.125 bits per weight) rotates each 128-weight block and stores one FP16 scale per block plus 16-level codes; the rotation spreads outliers so one scale per 128 weights is enough. The body is quantised with GPTQ on calibration text.
Chat template
The file embeds the model's chat template; no --chat-template-file is needed.