Qwen3.8-Flash-Next GSQ-RCO IQ3_XXS_M64_HC8

Qwen3.8-Flash-Next for a 64 GB Apple Silicon Mac: the full 262K context with MTP speculative decoding, vision and Neural Engine prefill, in 42.37 GiB of weights. It is ISTA-DASLab's GSQ-RCO IQ3_XXS first shard with two changes, at the same quality (KL against ISTA's best file, IQ3_S, 0.1795 against IQ3_XXS's 0.1834):

1. The routed down experts of layers 29, 30, 31 and 35 come from ISTA's Q2_0 file, byte for byte. That step alone is IQ3_XXS_M64.
2. The 194 low-rank projections of the hyper-connection mixers are Q8_0 instead of BF16. They are 1.2 of the 4.3 GiB a decode token reads, so it decodes about 10% faster.

It is made for ds4 Flash-Next M64, a fork of paperniuk/ds4 tuned for 64 GB M1 to M3 Max Macs, which has the Q8_0 hyper-connection kernels. Upstream ds4 loads this file through generic kernels: prefill runs at about half speed and decoding is no faster. With upstream, take IQ3_XXS_M64.

It needs two files from other repos:

• the n-gram table, ISTA's second shard (IQ3_XXS/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00002-of-00002.gguf, 28.8 GB). It is the same file in every ISTA quant and stays on the SSD; ds4 reads its rows as needed.
• for speculative decoding, the MTP block from paperniuk/Qwen3.8-Flash-Next-MTP-block (1.5 GB).

Run it

With the fork's launcher, which downloads all of it (76 GB with the vision encoder) and picks the context your Mac holds:

git clone https://github.com/pentaCoxian/ds4-flash-next.git
cd ds4-flash-next && make
./dstar pull --quant iq3m8
sudo sysctl iogpu.wired_limit_mb=57344   # once per boot, for 262K
./dstar serve                 # http://127.0.0.1:8010/v1, metrics on /dashboard

./dstar serve --dry-run prints the exact ds4-server command and environment, for running it without the launcher.

Memory

At 262K with MTP and the Neural Engine, ds4 plans 51.4 GiB: the weights and MTP block 43.8, the 8-bit KV cache 3.7, the prefill buffers 3.1 and the Neural Engine 0.9. The vision encoder maps only while it encodes an image. macOS lets the GPU use 48 GiB of 64 by default, which holds 32K; a limit of 53248 MiB holds 262K without the Neural Engine, 57344 MiB with it.

What changed

Against ISTA's IQ3_XXS:

The four down-expert layers are the ones that moved the KL against IQ3_S least when each of IQ3_XXS's 18 IQ4_NL layers was swapped to Q2_0 on its own; layer 47 alone added 0.07 on Japanese. Q8_0 uses ggml's reference rounding (blocks of 32, the scale is the block's absolute maximum / 127 stored as f16, codes rounded half away from zero); the relative RMS error of the requantized mixers is 0.66%. The GGUF metadata records the sources as general.mix.* and general.requant.*.

Quality

KL divergence against IQ3_S over six 2048-token sequences: C code, English prose and Japanese prose. The KL is a lower bound over IQ3_S's top 128 tokens plus the rest as one bucket.

Against IQ3_XXS_M64 itself the mean KL is 0.011 with 96.3% top-1 agreement: about as much as a different prefill chunking changes the outputs. At 4 bits (plain rounding) the mixers broke: KL 0.2585 against IQ3_S. No benchmark scores were run on this file; ISTA's scores for IQ3_XXS are a task average of 92.57 and LiveCodeBench 86.29.

Speed

M3 Max, 40-core GPU, the fork as of 2026-11-01, 40K context, 8K prompts (a coding agent / Japanese prose), 256 greedy tokens, best of three alternating rounds. Desktop applications were using the GPU, so runs move by 5 to 10%.

On GPU timestamps a decode token takes 20.6 ms with Q8_0 mixers against 22.4 ms with BF16 ones. Greedy decoding gives the same text with and without MTP. Only the M3 Max has been measured.

Rebuild it

From ISTA's IQ3_XXS and Q2_0 first shards, with gguf-tools/mixed/ from the fork:

python3 gguf-tools/mixed/gguf_mix.py splice \
  --base  Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf \
  --donor Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf \
  --tensors 'blk\.(29|30|31|35)\.ffn_down_exps\.weight' \
  --out   Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS_M64-00001-of-00002.gguf

python3 gguf-tools/mixed/gguf_requant_q8.py \
  --base Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS_M64-00001-of-00002.gguf \
  --out  Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS_M64_HC8-00001-of-00002.gguf

SHA-256 of this file and the files it is built from:

84bc308cb138ca04adab50bc1cadca017050a6514428e92af76fdc3cfd3d2569  Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS_M64_HC8-00001-of-00002.gguf
9c533e0650a672b4856eccbbbfd9a39a277988ada982e00129e7f19c8a8d4d2d  Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS_M64-00001-of-00002.gguf
219ea929900dfa9ef091f3aa473fdba6874b65fcb36526d7d851ac9e95856d15  Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf
69820c02ec7d0b45ef2ebb19d6620299db749fe2aded7f39f93c6b88b199b720  Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf

License

The weights are derived from Qwen/Qwen3.8-Flash-Next through ISTA-DASLab's GSQ-RCO quantization and are released under the same Qwen Community License 1.0. The full text is in LICENSE, and it must be included with any copy. Commercial products or services above 100M monthly active users or US$20M monthly revenue must display the model name prominently. See the license for the exact terms.