The reference checkpoint of qwen38-mlx-swift,
published so that nobody has to rebuild it: it is
Vontra/Qwen3.8-Flash-Next-MLX-4bit-MTP
with the routed experts re-quantized from 4-bit (group 32) to 3-bit (group 64).
Everything else — n-gram embedding tables, attention, shared expert, router gates,
norms, vision tower, native MTP block — is copied unchanged.
| Vontra 4-bit (source) | This pack | |
|---|---|---|
| Size on disk | 113 GB | 84 GB |
Routed experts (switch_mlp) |
4-bit, group 32 | 3-bit, group 64 |
| Other tensors | as converted by Vontra | identical |
| Weight shards | 22 | 22 (same names, same tensor-to-shard layout) |
Why: on a 96 GB Mac the 4-bit checkpoint barely fits; this one runs resident with room to spare, with the same greedy output on the project's regression guard.
| Value | |
|---|---|
| Greedy decode | 13.0–13.5 tok/s |
| Prefill, varied prose, 4.8 k tokens | 112 tok/s |
| Peak process memory, resident | 57.6–58.3 GB |
| First request (all layers loaded) | 48–63 s |
| Teacher-forced regression guard | 10/28 · −4.8003182, identical to the 4-bit source |
Full conditions: BENCHMARKS.md.
Requirements: Apple Silicon with 96 GB of unified memory, macOS 15+.
hf download VincentGOURBIN/Qwen3.8-Flash-Next-MLX-e3bit-MTP \
--local-dir ~/models/local/Qwen3.8-Flash-Next-MLX-e3bit-MTP
# with qwen38-mlx-swift
qwen38 serve --model-path ~/models/local/Qwen3.8-Flash-Next-MLX-e3bit-MTP --enable-thinking
The loader must support qwen4_exp and per-module quantization (the experts are
3-bit/g64, the rest 4-bit/g32, as declared in config.json).
Scripts/qwen4-exp-requantize-experts.py,
with mlx-vlm==0.6.17: shard by shard, every
*.mlp.switch_mlp.{gate_proj,up_proj,down_proj} tensor is dequantized from
4-bit/g32 and quantized to 3-bit/g64; every other tensor is copied byte for
byte into the same output shard. Run it yourself to reproduce this pack from
the Vontra source.
Derived from Qwen3.8 Flash-Next under the Qwen Community License 1.0
(included as LICENSE, unchanged from the source). Credit: Qwen for the model,
Vontra for the MLX conversion with native MTP. Note the licence's conditions
for very large commercial products and for "Model as a Service" or "AI Work
Assistant" businesses.
model-00006-of-00022.safetensors 5.3 GBmodel-00005-of-00022.safetensors 5.3 GBmodel-00004-of-00022.safetensors 5.3 GBmodel-00003-of-00022.safetensors 5.3 GBmodel-00002-of-00022.safetensors 5.3 GBmodel-00007-of-00022.safetensors 5.1 GBmodel-00001-of-00022.safetensors 4.3 GBmodel-00011-of-00022.safetensors 3.8 GBmodel-00016-of-00022.safetensors 3.8 GBmodel-00021-of-00022.safetensors 3.8 GBmodel-00019-of-00022.safetensors 3.8 GBmodel-00014-of-00022.safetensors 3.8 GBmodel-00013-of-00022.safetensors 3.8 GBmodel-00018-of-00022.safetensors 3.8 GBmodel-00008-of-00022.safetensors 3.8 GBmodel-00009-of-00022.safetensors 3.8 GBmodel-00012-of-00022.safetensors 3.5 GBmodel-00017-of-00022.safetensors 3.5 GBmodel-00015-of-00022.safetensors 3.5 GBmodel-00020-of-00022.safetensors 3.5 GBmodel-00010-of-00022.safetensors 3.5 GBmodel-00022-of-00022.safetensors 2.7 GB有文件？提交种子。
提交种子