Sign up

PocketAiHub/Qwen3.8-27B-MLX

🤗 Hugging Face sourceimage-text-to-textapache-2.0134 GBsafetensors✓ HF checksums availableupdated today
No torrent yet →

Qwen3.8-27B MLX

Official-source MLX releases of Qwen/Qwen3.8-27B, converted and validated by PocketAI Model Lab. This repository contains an experimental 2-bit AWQ build plus validated 4-bit, 6-bit, 8-bit, and unquantized BF16 variants derived from the same pinned official revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0.

Variants

Variant Folder Stored size Precision layout
MLX 2-bit AWQ (experimental) 2bit/ 11,038,990,758 bytes (10.28 GiB) 498 language modules affine Q2/group 32 with AWQ; vision tower BF16
MLX 4-bit 4bit/ 16,081,493,231 bytes (14.98 GiB) 498 language modules affine Q4/group 64; vision tower BF16
MLX 6-bit 6bit/ 22,804,831,851 bytes (21.24 GiB) 498 language modules affine Q6/group 64; vision tower BF16
MLX 8-bit 8bit/ 29,528,170,361 bytes (27.50 GiB) 498 language modules affine Q8/group 64; vision tower BF16
MLX BF16 bf16/ 54,740,455,716 bytes (50.98 GiB) Unquantized BF16 reference

Validation

The 4-bit, 6-bit, 8-bit, and BF16 variants passed the complete deterministic PocketAI feature suite:

  • 12/12 quality regression cases
  • 8/8 tool-calling cases
  • temporal video understanding (red->blue)
  • 4K-context needle retrieval (COBALT-7319)

The 4-bit variant additionally passed a 32K-context retrieval test at 32,770 actual prompt tokens.

The 2-bit build is intentionally marked experimental. It passed the text and image runtime smoke tests and exact 4K retrieval, but scored 11/12 on the compact quality gate, 3/8 on tool calling, and did not pass the synthetic temporal-video gate. Use 4-bit when instruction following, tools, or video understanding matter more than memory footprint.

4K MLX performance

These are warmed, single-run measurements on an Apple M5 Max with 128 GB unified memory using mlx==0.32.0, mlx-vlm==0.6.8, batch size 1, temperature 0, seed 0, and thinking disabled. The prompt contained 4,105 tokens and the exact retrieval response contained 9 generated tokens.

Variant Prompt prefill Generation End-to-end Peak MLX memory
MLX 2-bit AWQ (experimental) 541.7 tok/s 38.8 tok/s 7.82 s 16.00 GB
MLX 4-bit 733.1 tok/s 36.6 tok/s 5.86 s 21.80 GB
MLX 6-bit 584.7 tok/s 26.2 tok/s 7.38 s 29.54 GB
MLX 8-bit 600.1 tok/s 20.1 tok/s 7.30 s 37.27 GB
MLX BF16 623.8 tok/s 10.8 tok/s 7.43 s 58.29 GB

The full machine-readable evidence is in benchmarks/4k-feature-suite.json. These compact deterministic checks are regression gates, not a broad claim of quality parity across precisions.

Download and load

Install the validated runtime on an Apple Silicon Mac:

python -m pip install "mlx==0.32.0" "mlx-vlm==0.6.8"

Download only the desired variant and load its local subfolder:

from pathlib import Path

from huggingface_hub import snapshot_download
from mlx_vlm import generate, load
from mlx_vlm.prompt_utils import apply_chat_template

repo_id = "PocketAiHub/Qwen3.8-27B-MLX"
variant = "4bit"  # "2bit", "4bit", "6bit", "8bit", or "bf16"

snapshot = Path(snapshot_download(repo_id, allow_patterns=[f"{variant}/*"]))
model, processor = load(str(snapshot / variant))

prompt = apply_chat_template(
    processor,
    model.config,
    "Explain why seasons occur.",
    num_images=0,
    enable_thinking=False,
)
result = generate(
    model,
    processor,
    prompt,
    max_tokens=256,
    temperature=0.0,
    enable_thinking=False,
)
print(result.text)

For image input, pass an image path to mlx_vlm.generate and build the prompt with num_images=1. Video input was validated with MLX-VLM's video path and FPS arguments.

Reproducibility

  • Official source: Qwen/Qwen3.8-27B
  • Pinned source revision: 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
  • Converter: mlx-vlm==0.6.8
  • Base dtype: BF16
  • Quantization: MLX affine; 2-bit uses AWQ/group 32, while 4/6/8-bit use group 64
  • Every variant includes an artifact-manifest.json with exact file sizes and SHA-256 hashes.
  • release-manifest.json records the release inputs and validation evidence.

License and attribution

These conversions retain the original model's Apache 2.0 license. See LICENSE and the official Qwen model card.