Qwen3.8 27B — General & Cyber GGUF

Eight attributed downloads. One repository. Download only the file you need.

A preservation bundle of Q2_K, Q3_K_M, Q4_K_M and Q6_K for both general-purpose Orca Qwen and Cyber Qwen. The creators' weight bytes are unchanged. Mango adds source pins, checksums, hardware guidance and clearly scoped performance evidence.

Choose a download · Hardware guide · Speed and scores · All tests and results · Checksums · Provenance and notices

Join Infinity: model support, quant requests, benchmarks and updates.

Choose a download

All files are standalone text-model GGUFs. Q2 is the smallest storage option; Q4 is a practical starting point on a 24 GiB GPU; Q6 retains more precision. Those are format/storage recommendations, not measured quality rankings.

¹ Estimates assume short text context around 4K tokens, batch/concurrency 1, current llama.cpp and all main-model layers on the GPU. File size is a lower bound, not VRAM usage. KV cache, compute buffers, MTP, vision and desktop memory need extra headroom. See HARDWARE.md for runtime support, CPU/Apple guidance and caveats. A fit estimate is not a tested-device certification.

Read the Cyber lineage notice

cyber/v2/ contains philbert440's Cyber v2 Q4/Q6, including the exact Q6 weight file in Mango's current benchmark. That publisher does not ship Q2/Q3 in this release.

cyber/hfck99/ supplies the requested Q2/Q3 from mradermacher’s i1 importance-matrix release, whose declared conversion source is HFCK99/Qwen3.8-27B-Uncensored-Cyber (itself declaring philbert440's Aggressive model as its base). These are different declared checkpoint lineages. Do not interpret Q2→Q6 across those folders as the effect of quantization alone. All upstream cards and exact revisions are retained in provenance/.

Download one quant

python -m pip install -U huggingface_hub
hf download isilicon/mango-qwen38-27b-gguf \
  general/Qwen3.8-27B-Uncensored-Q4_K_M.gguf \
  SOURCE_MANIFEST.json CHECKSUMS.sha256 LICENSE NOTICES.md \
  --local-dir ./mango-qwen27

For Cyber v2 Q6, replace the weight path with:

cyber/v2/Qwen3.8-27B-Uncensored-Cyber-Q6_K.gguf

Use --revision <verified-commit> to pin a deployment; the publication receipt records the verified weight snapshot. Keep your chosen file's original path. Verify the downloaded file's SHA256 against CHECKSUMS.sha256; the list also covers files you did not choose to download.

Keep storage simple

The complete eight-weight collection is 130.66 GB / 121.69 GiB; the smallest individual file is 10.86 GB / 10.12 GiB. You normally need one, not all eight. Selective hf download avoids a full clone/download. Optional vision/MTP companions are linked below, rather than duplicated for every quant. Already quantized weights usually do not shrink much in ZIP archives.

HF publication uses server-side copies of the pinned source blobs, so it requires zero local weight downloads for the HF copy. This does not guarantee how Hugging Face accounts storage or provider-side deduplication. For local torrent staging, hard links reuse identical existing files on the same filesystem; new quant files still need their real bytes once. A torrent/magnet is tiny metadata, not an independent durable weight copy.

Hardware and runtime compatibility

Use a recent llama.cpp with the Qwen hybrid GDN / qwen35 architecture. Older runtimes may reject these files. These GGUF downloads are distinct from the MLX/TensorFold model. An MLX-only loader cannot directly load them.

Start with -c 4096, one request slot, and confirm a native load/generation before increasing context. Full advertised upstream context, every optional modality and every speculative path have not been validated for these eight files.

llama-server \
  --model ./mango-qwen27/general/Qwen3.8-27B-Uncensored-Q4_K_M.gguf \
  --ctx-size 4096 --parallel 1 --gpu-layers 99 --jinja \
  --host 127.0.0.1 --port 8081

The example follows the runtime's normal reasoning behavior and adds no artificial reasoning-token cap. CPU-only users can reduce GPU layers. Context memory and compatibility still require a local smoke test.

Optional companions

• General Orca vision: matching F16 projector, 0.93 GB. The publisher says its standard quants retain embedded MTP; enabling that path and obtaining a benefit requires compatible runtime support and a separate test.
• Cyber v2: preserved matching Q8 MTP head and Q8 projector, 3.164 GB + 0.629 GB. MTP is optional for normal text generation. Image input needs the matching projector and multimodal runtime.
• Cyber HFCK99 i1 lineage: this pinned imatrix release contains no vision projector. The documented use here is text generation. Projectors from other conversions have not been validated with these exact files; do not assume interchangeability.

Speed and quality evidence

Cyber v2 Q6_K with the separate Q8 MTP head scored 66.98/100 on the matched 202-question suite plus 12 mechanical agent cases. It took 45.60 recorded workload minutes, passed 12/12 mechanical cases, and recorded zero infrastructure errors or length finishes. This grade applies to that complete configuration. The other seven exact files remain ungraded by Mango.

The tested Q6 configuration trails general-purpose Qwen MLX4 + DFlash2 by 13.66 points (paired 95% interval [−18.91, −8.23]). These differ in checkpoint lineage, quantization, engine and draft, so the difference cannot be assigned to quantization alone. The local agent checks cover mechanical criteria; they do not establish complete semantic competence. Open-ended Cyber judge responses are collected but their independent Claude/Codex grading is pending.

For all 202 completed suite requests, the median end-to-end completion-token rate was 78.11 tokens/s among requests reporting more than 50 completion tokens. Median observed first-token latency was 0.81 seconds (202 recorded). Completion tokens are divided by complete request time, including prefill, thinking and API overhead; this is not isolated decode throughput. Workload minutes also include the 12 agent cases and exclude loading, restarts, wipes and idle time.

Test hardware: NVIDIA RTX PRO 6000 Blackwell Server Edition, 97,887 MiB VRAM, 160 GiB host RAM and 20 CPU cores. The run used 16 inference threads, llama.cpp 0.7.0-dev (build 1, commit 63b4b303b), initial 32K configured context, q8_0 K/V cache, concurrency 1, separate Q8 MTP with draft length 3, native medium thinking (API high mapped to medium), and vendor sampling. Stream context headers were absent; profile/preflight receipts establish the initial configured context. No maximum-context certification follows from this test.

PERFORMANCE.json contains the complete-run measurements, task/domain scores, definitions and source fingerprint. The evaluation archive provides per-question scores, all 66 paired comparisons, test definitions, scoring code and historical experiments. The exact Q6 mirror has further analysis and uncertainty tables.

No measured speed is available for the other seven files or other devices. General MLX4, Cyber IQ4_XS-imatrix and publisher BF16 scores do not transfer to these files. The Cyber Q2/Q3 releases also have a different declared lineage from Cyber v2 Q4/Q6.

The PirateFace torrent preserves the original verified weight/document snapshot at HF revision 010b68154e453a35eef22dd92ea8911fbb1b7663. These later HF documentation updates do not change its weight bytes or infohash.

Provenance, license and notices

Qwen is the base-model author. OrcaRouter, philbert440, HFCK99 and mradermacher retain their respective modification, conversion and publishing credits. isilicon / Mango preserves and documents these files; it does not claim to have trained, edited or quantized them. Redistribution follows Apache-2.0, with the full license and original publisher cards retained.

See NOTICES.md, LICENSE, SOURCE_MANIFEST.json and the unchanged upstream cards. Those cards contain the publishers' own disclaimers and benchmark claims; those claims are not independently reproduced Mango results. Source metadata/SHA equality is not a malware audit or a full independent restore/load of every file. Refusal editing does not establish answer correctness or suitability for deployment.