This file runs Qwen3.6-35B-A3B on a OnePlus Nord CE 2: a 2022 mid-range phone with a MediaTek Dimensity 900 (2× Cortex-A78 + 6× Cortex-A55), 7.4 GB of RAM of which about 3 GB is free, UFS 2.2 flash, Android 13, no root. Speeds, in real chat:
The file is 12.3 GB. About 1.3 GB of it stays in RAM; each layer keeps 80 of its 256 experts in RAM and reads the others from flash when the router picks them. Peak memory is about 4.9 GB.
From the 🤏 smolmaxxing project. The phone page is docs/phone.md and the full write-up is EXPERIMENT.md §22. The 80B sibling is hyperionaidev/Qwen3-Next-80B-A3B-phone-GGUF.
A standard GGUF. No custom tensors or keys, the whole output head, standard quant types (Q2_K, Q4_K). Any llama.cpp with Qwen3.5/3.6 MoE support (
qwen35moe) can load it. The speeds above need the smolmaxxing patch, whose expert pager keeps the model's experts on flash; without it the whole 12.3 GB is memory-mapped and a phone with ~3 GB free will be much slower.
| File | q36-q2-full.gguf, 12,348,766,912 bytes (12.3 GB) |
| SHA-256 | 723c881f8cbb11080fa4e500f4de53ab32d04aaebc8f497f629ff3ae425e5cd8 |
| Experts | Q2_K from mradermacher/Qwen3.6-35B-A3B-i1-GGUF (i1-Q2_K_S, imatrix), copied unchanged. That file keeps ffn_down_exps in Q4_K in 5 of the 40 layers. |
| Everything else | Q4_K: the non-expert tensors of mradermacher's i1-Q4_K_M requantized with his imatrix (--pure). In mradermacher's own Q2_K_S, much of the attention is Q2_K; here all of it is Q4_K, for 0.2 GB more. |
| Output head | all 248,320 rows. A 60K-row pruned head made the model write "palinate" for "palindrome", so it's kept whole (3 % slower on the phone). |
| Layout | experts last: the non-expert part comes first (1.3 GB), and each expert (gate, up, down) is about 1.03 MB |
| Context | 262,144 tokens in the metadata; run at 1,024 on the phone |
The file was never stored whole while it was built. tools/q36-phone.sh (HEAD=full) fetches the two sources' tensors with HTTP range requests, assembles the file in order and streams it to the phone over adb.
Generation speed in chat (2 prompts × 160 tokens, repeat penalty 1.15), 2 threads on the two A78 cores, 80 experts per layer in RAM, prefetch of the experts two layers ahead. Perplexity on the phone over 20 chunks of 128 tokens of a held-out English + code text (2,560 scored tokens).
| Cache bias | tok/s | Perplexity | vs unbiased | Cache misses |
|---|---|---|---|---|
| 0 (full quality) | 4.9 | 7.63 | ±0 | 10.8 % |
| 0.005 (recommended) | 5.7 | 7.68 | +0.7 % | 7.4 % |
| 0.01 | 6.0 | 7.78 | +2.0 % | 5.1 % |
Build the patched llama.cpp for Android as in the TV-box setup (static aarch64), push it with this file to /data/local/tmp/llm/, then:
cd /data/local/tmp/llm
# once: write the repack cache for the A55/A78 kernels (a one-token run)
GGML_REPACK_CACHE=$PWD/cache/q36f ./bin/llama-bench -m q36-q2-full.gguf -t 2 -C 0xC0 --cpu-strict 1 -p 0 -n 1 -r 1
# chat server
GGML_MOE_PAGER=1 GGML_REPACK_CACHE=$PWD/cache/q36f GGML_MOE_POOL=80 GGML_MOE_BIAS=0.005 \
GGML_MOE_PREFETCH_K=8 GGML_MOE_PREFETCH_AHEAD=2 GGML_MOE_READERS=8 \
./bin/llama-server -m q36-q2-full.gguf -c 1024 -np 1 --port 8090 --reasoning off -fa on \
-ctk q8_0 -ctv q8_0 -ub 4 -b 64 -t 2 -C 0xC0 --cpu-strict 1
-t 2 -C 0xC0 --cpu-strict 1 puts the two threads on the A78 cores (CPUs 6 and 7 on this chip): adding the slow cores made every setup slower. On another phone, use its big cores and size GGML_MOE_POOL to the free RAM (each step of 1 is about 41 MB).
Apache 2.0, as the original model.
Qwen/Qwen3.6-35B-A3B