Qwen/Qwen3.8-Flash-Next packed for sushi, a native inference engine for Apple Silicon. Target: a 48 GB Mac.
| part | how it is stored |
|---|---|
| routed experts (48 layers x 512, plus the MTP layer) | EXL3 at 2 bits per weight, made with our own converter |
| dense part (attention, GDN, shared experts, hyper-connections, indexer, embedding, lm_head) | 8-bit affine; the same as in Sushi-2.6bpw, Sushi-3bpw and Sushi-4bpw |
| n-gram embedding table | 4-bit affine, group size 32 (mlx-serve's quantization of the released bf16 table), one file ngram_table.bin, read from the SSD and never loaded into GPU memory; the same file as Sushi-2.6bpw and Sushi-3bpw |
| vision tower (Qwen3-VL ViT) | included: image and video input |
| MTP head | included |
| context | up to 1,048,576 tokens (config.json sets YaRN x4 over the native 262,144) |
| disk | ~65 GiB: 35.0 GiB of weights + 29.8 GiB n-gram table |
sushi only. This pack needs sushi v1.0.4 or later on Apple Silicon with macOS 26.2 or later. It does not load in transformers, vLLM, mlx-lm or exllamav3.
How to run it (install, launch flags, memory and context per Mac): sushi usage.
KLD against the bf16 checkpoint: 16 prompts x 512 tokens, scored to the first EOS (7186 positions), KV cache at kv8 (Sushi-2bpw: bf16 KV), every pack scored by sushi. Lower is better. The n-gram table is not counted: it stays on the SSD.
| pack | weights in GPU memory (GiB) | KLD | top-1 agreement |
|---|---|---|---|
| Sushi-4bpw | 63.68 | 0.0592 | 92.89% |
| oMLX oQ5e | 83.97 | 0.0625 | 92.40% |
| mlx-serve mixed-4-8bit | 70.13 | 0.0818 | 91.39% |
| Sushi-3bpw (4-bit n-gram table) | 49.33 | 0.1036 | 90.31% |
| Sushi-2.6bpw (4-bit n-gram table) | 43.95 | 0.1355 | 89.08% |
| oMLX oQ4e | 69.21 | 0.1370 | 88.87% |
| mlx-serve iQ-MLX 3.3bpw | 50.60 | 0.1987 | 86.28% |
| Vontra 4-bit g32 (TensorFold) | 75.60 | 0.2074 | 85.01% |
| Sushi-2bpw (4-bit n-gram table) | 34.97 | 0.2080 | 85.94% |
Sushi-2bpw is the smallest pack here and the only one that fits a 48 GB Mac. It comes within 5% of mlx-serve's iQ-MLX 3.3bpw KLD (0.208 vs 0.199) with 15.6 GiB less. Sushi-2.6bpw costs 9 GiB more for 35% lower KLD; take it if the Mac has 64 GB. Each pack is measured with the n-gram table it ships: 4-bit g32 in Sushi-2bpw, Sushi-2.6bpw, Sushi-3bpw and mlx-serve's packs, 4- or 5-bit in oMLX's, bf16 in Sushi-4bpw. The oMLX and mlx-serve packs were measured as published; the oMLX n-gram tables were repacked into sushi's file unchanged.
LLMProbe's Hard 50 set (MMLU-Pro, OlympiadBench, LiveBench, NIST Juliet): 42 correct, 7 incorrect, 1 partial (hit the token cap), 84.0%. llmprobe 0.6.12, xhigh effort, 12k reasoning / 16k output tokens, 8-bit KV and MTP.
The Sushi packs ship two copies of the same n-gram table, and either copy works with any of them. Use the 4-bit table to save 66 GiB of disk, or the bf16 table for a small quality gain. The table is read from the SSD, so GPU memory is the same either way.
| table | size on disk | ships with | KLD with Sushi-2.6bpw | KLD with Sushi-3bpw | KLD with Sushi-4bpw |
|---|---|---|---|---|---|
| 4-bit, group size 32 | 29.8 GiB | Sushi-2bpw, Sushi-2.6bpw, Sushi-3bpw | 0.1355 | 0.1036 | 0.0654 |
| bf16 | 95.4 GiB | Sushi-4bpw | 0.1303 | 0.1006 | 0.0592 |
To swap, download the other pack's ngram_table.bin into this pack's folder, replacing the current one:
hf download beamster/Qwen3.8-Flash-Next-Sushi-4bpw ngram_table.bin --local-dir <this pack's folder>
Then set ngram_table in config.json to match the file:
"ngram_table": {"file": "ngram_table.bin", "bits": 16, "group_size": 0} for the bf16 table.
LICENSE), including its conditions on large
commercial services.ngram_table.bin 32 GBmodel-trunk-00002-of-00002.safetensors 2.4 GBmodel-trunk-00001-of-00002.safetensors 2.4 GBmodel-embed.safetensors 675 MBmodel-lm-head.safetensors 656 MBmodel-exl3-L10-down.safetensors 213 MBmodel-exl3-L10-gate.safetensors 213 MBmodel-exl3-L11-down.safetensors 213 MBmodel-exl3-L11-gate.safetensors 213 MBmodel-exl3-L12-down.safetensors 213 MBmodel-exl3-L12-gate.safetensors 213 MBmodel-exl3-L13-down.safetensors 213 MBmodel-exl3-L13-gate.safetensors 213 MBmodel-exl3-L14-down.safetensors 213 MBmodel-exl3-L14-gate.safetensors 213 MBmodel-exl3-L15-down.safetensors 213 MBmodel-exl3-L15-gate.safetensors 213 MBmodel-exl3-L16-down.safetensors 213 MBmodel-exl3-L16-gate.safetensors 213 MBmodel-exl3-L17-down.safetensors 213 MBmodel-exl3-L17-gate.safetensors 213 MBmodel-exl3-L18-down.safetensors 213 MBmodel-exl3-L18-gate.safetensors 213 MBmodel-exl3-L19-down.safetensors 213 MBmodel-exl3-L19-gate.safetensors 213 MBmodel-exl3-L20-down.safetensors 213 MBmodel-exl3-L20-gate.safetensors 213 MBmodel-exl3-L21-down.safetensors 213 MBmodel-exl3-L21-gate.safetensors 213 MBmodel-exl3-L22-down.safetensors 213 MBmodel-exl3-L22-gate.safetensors 213 MBmodel-exl3-L23-down.safetensors 213 MBmodel-exl3-L23-gate.safetensors 213 MBmodel-exl3-L24-down.safetensors 213 MBmodel-exl3-L24-gate.safetensors 213 MBmodel-exl3-L25-down.safetensors 213 MBmodel-exl3-L25-gate.safetensors 213 MBmodel-exl3-L26-down.safetensors 213 MBmodel-exl3-L26-gate.safetensors 213 MBmodel-exl3-L27-down.safetensors 213 MBmodel-exl3-L27-gate.safetensors 213 MBmodel-exl3-L28-down.safetensors 213 MBmodel-exl3-L28-gate.safetensors 213 MBmodel-exl3-L29-down.safetensors 213 MBmodel-exl3-L29-gate.safetensors 213 MBmodel-exl3-L30-down.safetensors 213 MBmodel-exl3-L30-gate.safetensors 213 MBmodel-exl3-L31-down.safetensors 213 MBmodel-exl3-L31-gate.safetensors 213 MBmodel-exl3-L32-down.safetensors 213 MBmodel-exl3-L32-gate.safetensors 213 MBmodel-exl3-L33-down.safetensors 213 MBmodel-exl3-L33-gate.safetensors 213 MBmodel-exl3-L34-down.safetensors 213 MBmodel-exl3-L34-gate.safetensors 213 MBmodel-exl3-L35-down.safetensors 213 MBmodel-exl3-L35-gate.safetensors 213 MBmodel-exl3-L36-down.safetensors 213 MBmodel-exl3-L36-gate.safetensors 213 MBmodel-exl3-L37-down.safetensors 213 MBmodel-exl3-L37-gate.safetensors 213 MBmodel-exl3-L38-down.safetensors 213 MBmodel-exl3-L38-gate.safetensors 213 MBmodel-exl3-L39-down.safetensors 213 MBmodel-exl3-L39-gate.safetensors 213 MBmodel-exl3-L40-down.safetensors 213 MBmodel-exl3-L40-gate.safetensors 213 MBmodel-exl3-L41-down.safetensors 213 MBmodel-exl3-L41-gate.safetensors 213 MBmodel-exl3-L42-down.safetensors 213 MBmodel-exl3-L42-gate.safetensors 213 MBmodel-exl3-L43-down.safetensors 213 MBmodel-exl3-L43-gate.safetensors 213 MBmodel-exl3-L44-down.safetensors 213 MBmodel-exl3-L44-gate.safetensors 213 MBmodel-exl3-L45-down.safetensors 213 MBmodel-exl3-L45-gate.safetensors 213 MBmodel-exl3-L46-down.safetensors 213 MBmodel-exl3-L46-gate.safetensors 213 MBmodel-exl3-L47-down.safetensors 213 MBmodel-exl3-L47-gate.safetensors 213 MBmodel-exl3-L00-down.safetensors 213 MBmodel-exl3-L00-gate.safetensors 213 MBmodel-exl3-L00-up.safetensors 213 MBmodel-exl3-L01-down.safetensors 213 MBmodel-exl3-L01-gate.safetensors 213 MBmodel-exl3-L01-up.safetensors 213 MBmodel-exl3-L02-down.safetensors 213 MBmodel-exl3-L02-gate.safetensors 213 MBmodel-exl3-L02-up.safetensors 213 MBmodel-exl3-L03-down.safetensors 213 MBmodel-exl3-L03-gate.safetensors 213 MBmodel-exl3-L03-up.safetensors 213 MBmodel-exl3-L04-down.safetensors 213 MBmodel-exl3-L04-gate.safetensors 213 MBmodel-exl3-L04-up.safetensors 213 MBmodel-exl3-L05-down.safetensors 213 MBmodel-exl3-L05-gate.safetensors 213 MBmodel-exl3-L05-up.safetensors 213 MBmodel-exl3-L06-down.safetensors 213 MBmodel-exl3-L06-gate.safetensors 213 MBmodel-exl3-L06-up.safetensors 213 MBmodel-exl3-L07-down.safetensors 213 MBmodel-exl3-L07-gate.safetensors 213 MBmodel-exl3-L07-up.safetensors 213 MBmodel-exl3-L08-down.safetensors 213 MBmodel-exl3-L08-gate.safetensors 213 MBmodel-exl3-L08-up.safetensors 213 MBmodel-exl3-L09-down.safetensors 213 MBmodel-exl3-L09-gate.safetensors 213 MBmodel-exl3-L09-up.safetensors 213 MBmodel-exl3-L10-up.safetensors 213 MBmodel-exl3-L11-up.safetensors 213 MBmodel-exl3-L12-up.safetensors 213 MBmodel-exl3-L13-up.safetensors 213 MBmodel-exl3-L14-up.safetensors 213 MBmodel-exl3-L15-up.safetensors 213 MBmodel-exl3-L16-up.safetensors 213 MBmodel-exl3-L17-up.safetensors 213 MBmodel-exl3-L18-up.safetensors 213 MBmodel-exl3-L19-up.safetensors 213 MBmodel-exl3-L20-up.safetensors 213 MBmodel-exl3-L21-up.safetensors 213 MBmodel-exl3-L22-up.safetensors 213 MBmodel-exl3-L23-up.safetensors 213 MBmodel-exl3-L24-up.safetensors 213 MBmodel-exl3-L25-up.safetensors 213 MBmodel-exl3-L26-up.safetensors 213 MBmodel-exl3-L27-up.safetensors 213 MBmodel-exl3-L28-up.safetensors 213 MBmodel-exl3-L29-up.safetensors 213 MBmodel-exl3-L30-up.safetensors 213 MBmodel-exl3-L31-up.safetensors 213 MBmodel-exl3-L32-up.safetensors 213 MBmodel-exl3-L33-up.safetensors 213 MBmodel-exl3-L34-up.safetensors 213 MBmodel-exl3-L35-up.safetensors 213 MBmodel-exl3-L36-up.safetensors 213 MBmodel-exl3-L37-up.safetensors 213 MBmodel-exl3-L38-up.safetensors 213 MBmodel-exl3-L39-up.safetensors 213 MBmodel-exl3-L40-up.safetensors 213 MBmodel-exl3-L41-up.safetensors 213 MBmodel-exl3-L42-up.safetensors 213 MBmodel-exl3-L43-up.safetensors 213 MBmodel-exl3-L44-up.safetensors 213 MBmodel-exl3-L45-up.safetensors 213 MBmodel-exl3-L46-up.safetensors 213 MBmodel-exl3-L47-up.safetensors 213 MBmodel-exl3-mtp-L00-down.safetensors 213 MBmodel-exl3-mtp-L00-gate.safetensors 213 MBmodel-exl3-mtp-L00-up.safetensors 213 MBmodel-mtp.safetensors 96 MBmodel-00047.safetensors 375.6 kBmodel-00018.safetensors 316.2 kBmodel-00051.safetensors 316.2 kBmodel-00039.safetensors 298.8 kBmodel-00083.safetensors 298.8 kBmodel-00081.safetensors 296.2 kBmodel-00071.safetensors 296.2 kBmodel-00063.safetensors 296 kBmodel-00010.safetensors 295.6 kBmodel-00037.safetensors 295.6 kBmodel-00045.safetensors 295.6 kBmodel-00055.safetensors 295.6 kBmodel-00026.safetensors 295.6 kBmodel-00089.safetensors 295.6 kBmodel-00095.safetensors 295.6 kBmodel-00012.safetensors 293.5 kBmodel-00014.safetensors 293.5 kBmodel-00020.safetensors 293.5 kBmodel-00022.safetensors 293.5 kBmodel-00031.safetensors 293.5 kBmodel-00033.safetensors 293.5 kBmodel-00041.safetensors 293.5 kBmodel-00057.safetensors 293.5 kBmodel-00059.safetensors 293.5 kBmodel-00065.safetensors 293.5 kBmodel-00067.safetensors 293.5 kBmodel-00073.safetensors 293.5 kBmodel-00075.safetensors 293.5 kBmodel-00077.safetensors 293.5 kBmodel-00085.safetensors 293.5 kBmodel-00003.safetensors 293.5 kBmodel-00091.safetensors 293.5 kBmodel-00097.safetensors 293.5 kBmodel-00016.safetensors 292.7 kBmodel-00006.safetensors 288.8 kBmodel-00087.safetensors 231.3 kBmodel-00069.safetensors 211 kBmodel-00028.safetensors 210.9 kBmodel-00008.safetensors 210.7 kBmodel-00024.safetensors 210.7 kBmodel-00035.safetensors 210.7 kBmodel-00043.safetensors 210.7 kBmodel-00053.safetensors 210.7 kBmodel-00061.safetensors 210.7 kBmodel-00079.safetensors 210.7 kBmodel-00049.safetensors 210.6 kBmodel-00093.safetensors 210.6 kBmodel-00001.safetensors 206.2 kBmodel-00005.safetensors 169.9 kBmodel-00099.safetensors 108 kBmodel-00029.safetensors 82.7 kBHave the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo beamster/Qwen3.8-Flash-Next-Sushi-2bpw ./model-folderQwen/Qwen3.8-Flash-Next