Qwen3.8-27B-MixQ4A6E8-v0 (experimental rc1)

Experimental candidate — not optimized, recommended, or generally validated. This immutable candidate exists to make a measured mixed-quant experiment reproducible. No quality-equivalence, speed-superiority, memory, or maximum-context claim is made beyond the sealed evidence cited below.

This community release is not affiliated with or endorsed by Qwen or Alibaba Cloud.

Artifact

Verify the download with sha256sum -c SHA256SUMS. MANIFEST.json is the canonical machine-readable provenance record.

Recipe and immutable provenance

• Upstream model: Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 (Apache-2.0).
• BF16 input: ggml-org/Qwen3.8-27B-GGUF@0669b98607d47046c7c2b3f801011d54a08cfccf, SHA256 5a3eedc837bcbd1365cdbf5b71e698df3122e76586ba872f07ce3ed4a9bfa97e.
• Quantizer: llama.cpp b10442, commit 9b0a2ce859d3884252705e9ac7c93c98616bb238.
• External imatrix: Bartowski Qwen3.8-27B-imatrix.gguf@f0eec4a4bb4975114a030d048952d83c0a53c034, SHA256 aaa933d4b9ce23e1f65c548ad34f16956d8af44a51b5c15bf4f393ba59508cd8.
• Quantization: Q4_K_M fallback; Q6_K for matched .*attn.* and .*ssm.* tensors; Q8_0 token embedding and output. Tensor rules SHA256 4000c7ce7f467c4972df085e0ce115bb1f79cc252be8dbc2dd28b14eea2599e4.

The external importance matrix was not created by this project. See ATTRIBUTION.md. Build evidence (dry-run validation, quantize log, commands, environment): results/build-mixq4a6e8-v0-20260818T231742Z.

Sealed Q4_K_M comparison (quality lane)

Full WikiText-2 test set (602 chunks, ctx/batch/ubatch 512, pinned corpus, BF16-relative KL against freshly generated BF16 base logits), pinned b10442 Vulkan lane, identical protocol across artifacts. Lower is better.

Within this sealed two-artifact comparison, the candidate's BF16-relative KL is 18.2492% lower than Q4_K_M (0.007835 versus 0.009584); the gap is ~35× the combined standard error. Its full PPL is 0.0091 lower (6.6830 versus 6.6921); given the ±0.04 uncertainties this supports the KL claim but only a weak PPL direction. Evidence: results/ppl-full-bf16ngl48-mixq4a6e8-v0-vulkan-b10442-20260819T063633Z.

Sealed Q4_K_M comparison (speed lanes)

Identical llama-bench protocol on both lanes: pp512/pp2048/tg128, 5 repetitions, pinned b10442 CUDA build, medians in tok/s. Higher is better.

Single RTX 3090 24 GiB (one GPU, tensor split 1):

Candidate deltas: −2.3681% (pp512), −2.3206% (pp2048), −4.8880% (tg128). Evidence: results/llama-bench-mixq4a6e8-v0-single-gpu-cuda-b10442-20260818T233255Z.

Dual RTX 3090 (layer split 1/1):

Candidate deltas: −2.7386% (pp512), −4.4062% (pp2048), −4.6733% (tg128). Evidence: results/llama-bench-frontier-q4toq8-cuda-b10442-20260819T094256Z.

Net measured trade within these sealed comparisons: the candidate is 7.0291% larger than Q4_K_M, 18.2492% lower in BF16-relative KL, and 2.3–4.9% slower across the three workloads. Whether that trade serves a given use case is left to the reader; no recommendation is made.

Measured Q4–Q8 frontier (CUDA lane, dual RTX 3090)

Six artifacts, identical sealed protocol; KL/PPL from the pinned Vulkan perplexity lane (Q8_0 not KL/PPL-measured here). Medians.

Within this sealed set no artifact is simultaneously higher-fidelity (KL) and faster (tg128) than another: every point is nondominated. MixQ4A6E8-v0 occupies the measured intermediate position between Q4_K_M and Q5_K_M — 1.0883% smaller and +1.0672%/+1.5420%/+2.5225% faster (pp512/pp2048/ tg128) than Q5_K_M at worse fidelity (KL 0.007835 versus 0.003949).

This does not establish a broad optimization, best, recommendation, VRAM benefit, Q8/BF16 equivalence, or industry-benchmark quality claim.

Validation scope: what is NOT covered

• MTP (multi-token prediction draft heads): not validated for this artifact; do not assume the sibling's MTP parity results transfer.
• Multimodal projector: not validated for this artifact.
• Text integration smoke suite (chat template, thinking controls, tool calling): passed for this artifact (exact tokenizer/architecture metadata parity plus five deterministic integration tests; sealed run text-validation-mixq4a6e8-v0-vulkan-b10442-20260819T131943Z). It is a reproducibility smoke, not a quality benchmark.
• Downstream task-quality suites: not run. Perplexity/KL are distributional proxies, not task benchmarks.
• Downstream task-quality suites: not run. Perplexity/KL are distributional proxies, not task benchmarks.

Compatibility and limitations

• Evidence was collected on a dedicated validation host with 2× NVIDIA RTX 3090 24 GiB GPUs: pinned b10442 CUDA sm_86 build (single- and dual-GPU speed lanes) and pinned b10442 Vulkan tools (quality lane). Other runtimes and hardware are unverified.
• At 19,021,793,888 bytes the artifact fits a single 24 GiB card with context headroom; exact maximum context per runtime is unmeasured and not claimed.
• The tensor-selection rules remain experimental even though this exact artifact was built and experimentally checked.
• Review MANIFEST.json, NOTICE, and ATTRIBUTION.md before redistribution.

Sibling release

MixQ5A6E8-v0 (Q5_K_M fallback, same attention/SSM and embedding rules): Alogotron/Qwen3.8-27B-MixQ5A6E8-GGUF.