Qwen3.8-Flash-Next GSQ-RCO IQ3_XXS_M64

An IQ3_XXS file of Qwen3.8-Flash-Next that fits a 64 GB Apple Silicon Mac at the full 262K context with room for the Neural Engine. It is ISTA-DASLab's GSQ-RCO IQ3_XXS first shard with the routed down experts of four layers taken from the same release's Q2_0 file. Nothing was requantized: every tensor is a byte-for-byte copy of one of the two ISTA files. Weights: 42.92 GiB instead of 43.80.

With ds4 Flash-Next M64, take IQ3_XXS_M64_HC8 instead. It is this file with the hyper-connection mixers in Q8_0: the same quality, 0.55 GiB smaller and about 10% faster decoding with that build's Q8_0 kernels. This file is for builds without them, such as upstream paperniuk/ds4, which run the HC8 file's mixers at half the prefill speed.

It needs two files from other repos:

• the n-gram table, ISTA's second shard (IQ3_XXS/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00002-of-00002.gguf, 28.8 GB). It is the same file in every ISTA quant and stays on the SSD; ds4 reads its rows as needed.
• for speculative decoding, the MTP block from paperniuk/Qwen3.8-Flash-Next-MTP-block (1.5 GB).

Run it

With ds4 Flash-Next M64's launcher, which downloads all of it (77 GB with the vision encoder) and picks the context your Mac holds:

git clone https://github.com/pentaCoxian/ds4-flash-next.git
cd ds4-flash-next && make
./dstar pull --quant iq3m
./dstar serve --quant iq3m    # http://127.0.0.1:8010/v1

./dstar serve --dry-run prints the exact ds4-server command and environment, for running it without the launcher or with another ds4 build.

What changed

The GGUF metadata records the source files and the selection as general.mix.base, general.mix.donor and general.mix.donor_tensors.

How the layers were chosen

IQ3_XXS stores the routed down experts of 18 layers in IQ4_NL and of the other 30 in Q2_0. Each of the 18 was swapped to Q2_0 on its own, and the KL divergence against IQ3_S (ISTA's best file) was measured over six 2048-token sequences: C code, English prose and Japanese prose. Layers 29, 30, 31 and 35 moved it least, within the noise of the measurement; layer 47 alone added 0.07 on Japanese.

The KL is a lower bound over IQ3_S's top 128 tokens plus the rest as one bucket. No benchmark scores were run on this file; ISTA's own scores for the IQ3_XXS file are a task average of 92.57 and LiveCodeBench 86.29.

Memory and speed

On a 64 GB Mac with sudo sysctl iogpu.wired_limit_mb=57344, ds4 plans 51.9 GiB at 262K with MTP and the Neural Engine: the weights and MTP block 44.3, the 8-bit KV cache 3.7, the prefill buffers 3.1 and the Neural Engine 0.9. The vision encoder maps only while it encodes an image.

M3 Max, 40-core GPU, 262K context under an emulation of a 64 GB Mac, 8K prompts (a coding agent and Japanese prose), greedy decoding with MTP:

Rebuild it

With gguf-tools/mixed/gguf_mix.py from ds4 Flash-Next M64:

python3 gguf-tools/mixed/gguf_mix.py splice \
  --base  Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf \
  --donor Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf \
  --tensors 'blk\.(29|30|31|35)\.ffn_down_exps\.weight' \
  --out   Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS_M64-00001-of-00002.gguf

SHA-256 of this file and its sources:

9c533e0650a672b4856eccbbbfd9a39a277988ada982e00129e7f19c8a8d4d2d  Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS_M64-00001-of-00002.gguf
219ea929900dfa9ef091f3aa473fdba6874b65fcb36526d7d851ac9e95856d15  Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf
69820c02ec7d0b45ef2ebb19d6620299db749fe2aded7f39f93c6b88b199b720  Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf

License

The weights are derived from Qwen/Qwen3.8-Flash-Next through ISTA-DASLab's GSQ-RCO quantization and are released under the same Qwen Community License 1.0. The full text is in LICENSE, and it must be included with any copy. Commercial products or services above 100M monthly active users or US$20M monthly revenue must display the model name prominently. See the license for the exact terms.