Qwen3.8-27B-Escha-W2 — hipfire builds
EschaLabs' Escha-W2 2-bit trellis quantization of Qwen3.8-27B, packaged for hipfire on AMD RDNA3/RDNA4.
The trellis codes are stored verbatim and decoded inside the GEMV — there is
no decode-at-load step and no re-quantisation of the coded projections. All
three builds below contain byte-identical coded weights; they differ only in
how the dense tensors (embeddings, lm_head, biases, the F16 GatedDeltaNet
in_proj_a/in_proj_b) are stored.
All three also carry upstream's multi-token-prediction head — 15
mtp.* tensors, 849 MB at F16, byte-identical across the three builds and
included in the sizes below. hipfire loads it for speculative decode
(measured accept rate 0.568, ~2.7 tokens committed per verify window).
Enabling it changes greedy output; see Speculative decode below.
This is the DENSE sibling of
qwen3.6-35b-a3b-escha.
Unlike the 35B's fused MoE gate_up, the 27B's gate_proj is K=2 while
up_proj and down_proj are K=3, so they cannot share a kernel call.
Builds
| file | size | resident | prefill | decode | PPL | KLD vs -pro |
|---|---|---|---|---|---|---|
qwen3.8-27b.escha-xt |
10.45 GB | 11.89 GB | 122 tok/s | 12.3 tok/s | 9.7242 | 0.008943 |
qwen3.8-27b.escha |
10.77 GB | 12.22 GB | 119 tok/s | 12.1 tok/s | 9.6753 | 0.000534 |
qwen3.8-27b.escha-pro |
11.16 GB | 12.66 GB | 113 tok/s | 10.8 tok/s | 9.6486 | 0.000000 |
The unsuffixed build is the default. Its dense tensors are MQ6: +12%
decode over -pro for +0.28% perplexity and a KLD of 0.000534 — the
down-quant is close to free, so there is little reason to pay for -pro
unless you want the most faithful build available.
-pro keeps the dense tensors at Q8_0, a bit-exact repack of Escha's
own per-row int8 into hipfire's per-32-block form (the row scale replicated
into every block). It is the KLD reference the other rows are measured
against; the 0.000000 is a negative control, not a rounding.
-xt is 0.3 GB smaller again and the fastest, at +0.78% perplexity.
-xt/base/-prodescribe the DENSE tensors only. All three builds carry byte-identical 2-bit trellis codes and a byte-identical F16 MTP head; they differ solely in how embeddings, lm_head, biases and the F16 GatedDeltaNet projections are stored. Do not read these as bit-widths —qwen3.8-27b.mq4is a genuinely 4-bit 15.66 GB model and is a different thing entirely.
Usage
hipfire run qwen3.8-27b.escha "What is the capital of France?"
Needs a hipfire build with escha support; the codes are decoded inside the
GEMV, so there is no conversion step and no dequantised second copy in memory.
The registry tags (qwen3.8:27b-escha, -pro, -xt) resolve once escha
support reaches a release — until then, point it at the file.
Where it sits against the non-escha ladder
Same slice, same harness, same box
(hipfire-models/qwen3.8-27b):
| build | size | PPL | decode |
|---|---|---|---|
qwen3.8-27b.mq6 |
21.75 GB | 9.0042 | 9.5 tok/s |
qwen3.8-27b.escha-pro |
11.16 GB | 9.6486 | 10.8 tok/s |
qwen3.8-27b.escha |
10.77 GB | 9.6753 | 12.1 tok/s |
qwen3.8-27b.escha-xt |
10.45 GB | 9.7242 | 12.3 tok/s |
qwen3.8-27b.mq3 |
12.62 GB | 10.0643 | 15.7 tok/s |
Every escha build beats mq3 on quality while being smaller, and beats
mq6 on both speed and size at +7% perplexity. Against mq3 it trades
tokens/sec for quality-per-byte: pick escha if VRAM or fidelity is the
constraint, mq3 if throughput is.
Decode speed, and why it is what it is
mq3 and mq6 reach
~198–207 GB/s on this box, essentially the memory ceiling — they are
bandwidth-bound, so smaller directly means faster. Escha reaches 121–130 GB/s
across the three builds because it also decodes the trellis — 7 arithmetic ops per weight. Ablated
on the shipped GEMV, removing that arithmetic entirely is 1.83× on that
kernel; the escha GEMVs are roughly a third of a decode step, so the
end-to-end ceiling is nearer 12–13 tok/s. That cost is what buys the 2-bit
residency, and a codebook lookup table does not avoid it: the state differs
per lane, so the table load is a gather and measures exactly the same as the
arithmetic it replaces.
Speculative decode (MTP)
The head is loaded automatically; HIPFIRE_MTP_MODE=off disables it.
HIPFIRE_MTP_ACCEPT_STATS=1 prints the running accept rate.
It changes greedy output. Speculative decode here is not
output-identical to running without it: the verify step evaluates the
trunk over a batch of positions with different kernels than single-token
decode, and those differ at ULP level, which flips occasional near-ties.
This is a property of hipfire's MTP path, not of this model — a
.mtp-sidecar model on a different base and quant diverges the same way.
Both streams are valid greedy samples of the same weights; if you need one
exact reproducible stream, pin HIPFIRE_MTP_MODE.
How these were measured
Measured on gfx1151 (Strix Halo, 40 CU, ~209–220 GB/s achievable), hipfire at
bb77ff87d. PPL and KLD on a fixed wikitext-2 slice
(benchmarks/quality-baselines/slice, md5 83b0205a…), 1020 scored tokens,
per-token scoring with unquantised KV. KLD is against the -pro build.
Prefill and decode at a 2k prompt, greedy, median of repeated runs.
Resident is a measured GTT delta against an idle baseline (~3.4 GB), taken
with an 8192-token q8 KV cache — so it includes the KV, not just the weights.
Waiting for GTT to fall back to idle between builds matters: measured
back-to-back without that, the second and third models appear to need 0.5 GB.
PPL here is a relative measure for comparing these three builds. Do not compare it to a perplexity from a different corpus, context length or KV precision — including the sibling 35B card, which uses a different slice geometry, and which is a different base model besides.
Provenance and licence
- Base model: Qwen/Qwen3.8-27B
- Quantization: EschaLabs/Qwen3.8-27B-Escha-W2
(
quant_method: escha, format 2.0). The W2 trellis codes here are EschaLabs' work, redistributed unmodified —hipfire-quantize --format eschapasses the coded payload through byte-for-byte and repacks only the dense tensors.HIPFIRE_ESCHA_DENSEselects that recipe:mq6for the default build,q8for-pro,mq4v2for-xt. - Apache-2.0, following upstream.
The container is hipfire's .hfq under an .escha extension. The -xt /
-pro variant suffix names the dense recipe, which is the only thing that
differs between the three.