Qwen3.8-27B mixed-quantization GGUF — legacy research release

Status update — 2026-10-22: legacy; not recommended for new deployments. This release is retained as a record of a single-GPU quantization and inference experiment. It is no longer presented as a current model or performance recommendation. The figures below are historical, author-reported measurements, not current rankings or performance guarantees. No new inference benchmark was run for this documentation update. There is no verified like-for-like comparison here establishing that a particular newer model outperforms this artifact.

What this repository contains

This is a community quantization of Qwen3.8-27B, derived from the Unsloth GGUF distribution. It is not a newly trained foundation model, a fine-tune, an official Qwen release, or an official Unsloth quantization. The documented experiment used an AMD RX 9070 XT with 16 GB VRAM, Windows, and llama.cpp's Vulkan backend.

• Task: text generation. This repository does not establish working image or video inference; the upstream model's broader capabilities are not validation of this GGUF configuration.
• Format: GGUF, approximately 2.72 effective bits per parameter overall.
• Quantization: a mixed tensor map using Q2_K, IQ2_XXS, IQ3_XXS, IQ4_XS, Q3_K, Q4_K, Q8_0, and F32; see the published tensor map. It is not uniformly 2-bit, and the earlier description "IQ2_XXS elsewhere" omitted these exceptions.
• Context: the recorded configuration allocated 262,144 tokens (256 Ki). Allocated capacity is different from occupied KV-cache depth and does not establish reliable performance across the entire context window.

Filename warning: pareto-bf16.gguf is the historical filename of the quantized artifact. The original build notes describe BF16 source weights; the published output is mixed-quantized, not a full-BF16 release. A bounded inspection of the published GGUF header found 866 tensors and no BF16 tensors, matching the mixed-type map above. The Hub currently displays a BF16/16-bit label; that label is misleading. The original filename, download URL, and checksum are retained for existing references.

Intended use and limitations

Appropriate uses are studying the archived measurements, inspecting the tensor allocation recipe, and undertaking independent replication. This release is not recommended as a default for new deployments or as a production-ready performance configuration. Compatibility, output quality, memory use, and throughput need fresh evaluation on the intended hardware and software.

For new evaluations, consult the maintained Qwen organization page, Unsloth distributions, and upstream llama.cpp. These are starting points for evaluation, not benchmarked replacements endorsed by this repository. No direct successor to this artifact is designated here.

Historical results — read with their conditions

The source of record is the linked GitHub evidence repository at revision c7abad32e4f2465617da890429831ddf534795db. The table preserves reported measurements; it does not assert independent replication or comparability with current model leaderboards.

A later check did not reproduce the headline speeds. The evidence repository records 23.0–29.3 tokens/s across six empty/loaded-cache shallow probes and a separate clean-cell mean of 27.2 tokens/s (six probes spanning 25.2–30.5). These post-publication checks were unresolved/UNCLASSIFIED. The record leaves open whether this reflects a slower hardware/driver state or conditions that could not be reproduced. See the replication disclosure, results/protocol-vs-state-2026-08-24.json, and results/working-depth-clean-cell-2026-08-24.json.

Evaluation and reproduction boundaries

• Server measurements with multi-token prediction/speculative decoding enabled and llama-bench raw decoding with MTP disabled are different regimes. Their tokens/s figures should not be combined into a single speed claim.
• The published records do not support a blanket statement that every comparison was interleaved. The paper discusses sequential arms, possible order effects, driver-state variation, and sustained drift.
• Resident depth in the server checks is author-recorded; the saved result JSONs do not provide a complete independent audit of the cache-fill procedure.
• A small depth-GSM8K sample and a repeated retrieval prompt do not establish general long-context reasoning or task quality. The published record does not contain a completed RULER evaluation. No new evaluation is promised here.
• The exact diff for the binary used in the selective-attention measurements was not preserved. The supplied selector patch is an incomplete reconstruction, with unfinished selector/mask logic. It is not a verified, behavior-equivalent implementation of the measured binary.
• The reproduction scripts contain manual or unimplemented steps. A tensor map, historical command line, and published checksum do not by themselves guarantee a byte-identical rebuild or reproduction of the reported speed.
• Other GPU architectures, operating systems, current llama.cpp revisions, and general production workloads were not validated by this documentation audit.

Artifact identity and historical launch notes

The Hub records one model file:

• File: pareto-bf16.gguf
• Size: 9,298,488,288 bytes, approximately 8.66 GiB.
• Published SHA256: 57dda505ecd2b0731341c002ff03ede1526740dc899e6f840ae1dfb7f1b3fa81.
• Tensor map, artifact notes, and historical launch flags.

The SHA256 is the published artifact identifier, not a claim that this documentation audit downloaded and rehashed the entire model. Verify a downloaded copy before using it. Historical launch examples are retained for research and must not be read as "works everywhere" instructions or present-day compatibility guarantees. Upstream llama.cpp does not gain the custom selector merely by setting the documented environment variables.

Project note

This repository preserves the available experimental results and configuration notes for research reference. The limitations above describe gaps in the archived record. Feedback can be shared through the repository's Community tab.

Credits and licenses

• Base model: Qwen3.8-27B, Apache-2.0 license.
• Conversion/quantization lineage: Unsloth's GGUF distribution and llama.cpp. This artifact is independently published by Kindadodgy.
• The experimental selective-attention work was described as drawing on Quest/NSA-style research; this attribution is not an implementation-validation claim.
• The original publication disclosed human oversight and AI-assisted implementation and analysis.
• Model weights retain their declared Apache-2.0 license. Recipe/configuration code retains the GitHub repository's MIT license; paper text and figures retain their declared CC-BY-4.0 license. This status update does not change those licenses or remove the artifact.