Qwen3.8-9B Distill MLX

Unofficial MLX conversions of the third-party distill empero-ai/Qwen3.8-9B, pinned to revision 0934f3d2327ff2df2197495278c4c46ae5a56bd9. The source is a third-party full-parameter distillation based on Qwen/Qwen3.5-9B; it is not an official Qwen3.8 release. Conversion and validation were performed by PocketAI Model Lab.

Variants

Precision	Folder	Packaged size
4-bit	`4bit/`	5.57 GiB
8-bit	`8bit/`	9.74 GiB
BF16	`bf16/`	17.55 GiB

The 4-bit and 8-bit variants use MLX affine quantization with group size 64. The vision tower remains BF16. The BF16 variant is unquantized. Native source MTP tensors are intentionally excluded from all three standard MLX builds.

Validation

All three variants passed deterministic 4K retrieval and local runtime tests.
The 4-bit build passed 12/12 quality cases, 8/8 tool-calling cases, temporal video understanding, and exact retrieval at 65,545 formatted tokens.
The BF16 build passed the complete deterministic feature suite at 4K.
The configured context window is 262,144 tokens; only the tested lengths above are claimed here.

Local 4K performance

Warmed single-run measurements on an Apple M5 Max with 128 GB unified memory, mlx==0.32.0, mlx-vlm==0.6.8, batch size 1, temperature 0, seed 0, and thinking disabled:

Precision	Prefill tok/s	Decode tok/s	Peak MLX memory
4-bit	3197.3	100.28	6.97 GB
8-bit	3144.8	57.86	11.37 GB
BF16	3103.4	30.56	19.85 GB

These compact deterministic checks are release regression gates, not broad claims of benchmark quality or parity across precisions. Exact evidence hashes are in each variant's validation-summary.json and artifact-manifest.json.

Download and load

python -m pip install "mlx==0.32.0" "mlx-vlm==0.6.8"

from pathlib import Path

from huggingface_hub import snapshot_download
from mlx_vlm import generate, load
from mlx_vlm.prompt_utils import apply_chat_template

repo_id = "PocketAiHub/Qwen3.8-9B-MLX"
variant = "4bit"  # "4bit", "8bit", or "bf16"
snapshot = Path(snapshot_download(repo_id, allow_patterns=[f"{variant}/*"]))
model, processor = load(str(snapshot / variant))
prompt = apply_chat_template(
    processor,
    model.config,
    "Explain why seasons occur.",
    num_images=0,
    enable_thinking=False,
)
result = generate(
    model,
    processor,
    prompt,
    max_tokens=256,
    temperature=0.0,
    enable_thinking=False,
)
print(result.text)

Image and video inputs use the normal mlx_vlm.generate media arguments.

Reproducibility and limitations

Source: empero-ai/Qwen3.8-9B at 0934f3d2327ff2df2197495278c4c46ae5a56bd9
Declared base: Qwen/Qwen3.5-9B
Standard MLX conversion intentionally excludes native MTP tensors
This is an experimental community release; verify behavior for your use case

License and attribution

The source repository declares Apache-2.0. This derivative includes the Apache 2.0 text in LICENSE. Original model credit remains with Empero and the Qwen team; PocketAI is the conversion publisher.