BaseQuant V1 build of Qwen/Qwen3.6-35B-A3B, as a .base bundle for baseRT and a GGUF for llama.cpp.
| Size | baseRT bundle | GGUF (llama.cpp) |
|---|---|---|
| Medium | Qwen3.6-35B-A3B-BaseQuant-V1-Medium.base (23.08 GB) |
Qwen3.6-35B-A3B-BaseQuant-V1-Medium.gguf (22.13 GB) |
The .base bundle also carries the vision tower and the MTP head for speculative decoding. It needs baseRT 0.5.2 or later.
# baseRT
hf download basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 Qwen3.6-35B-A3B-BaseQuant-V1-Medium.base --local-dir .
basert serve ./Qwen3.6-35B-A3B-BaseQuant-V1-Medium.base
# SuperFluid
curl -fsSL https://superfluid.sh/install.sh | sh
superfluid serve basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 --runtime basert --pull-file Qwen3.6-35B-A3B-BaseQuant-V1-Medium.base
# llama.cpp
llama-server -hf basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 -hff Qwen3.6-35B-A3B-BaseQuant-V1-Medium.gguf -c 32768 -ngl 99
KL divergence of the next-token distribution from the BF16 model, on held-out text: 58 prompts in ten categories, 32,507 scored tokens. None of it was used to build these files. Lower is closer to BF16. The .base bundle is scored in baseRT, and the GGUFs in llama.cpp. unsloth's UD-Q4_K_M, the same size, is shown for reference.
| Category | Tokens | BaseQuant .base |
BaseQuant GGUF | unsloth UD-Q4_K_M |
|---|---|---|---|---|
| Agentic coding | 2,228 | 0.00465 | 0.00490 | 0.00515 |
| Chat | 3,342 | 0.00648 | 0.00634 | 0.00645 |
| Code | 2,883 | 0.00427 | 0.00434 | 0.00450 |
| Long context | 2,048 | 0.00427 | 0.00438 | 0.00536 |
| Math | 4,608 | 0.00304 | 0.00318 | 0.00265 |
| Multilingual | 4,232 | 0.00222 | 0.00213 | 0.00233 |
| Prose | 3,994 | 0.00624 | 0.00587 | 0.00628 |
| Research | 2,123 | 0.00737 | 0.00848 | 0.00584 |
| Science | 4,382 | 0.00467 | 0.00510 | 0.00490 |
| Tool use | 2,667 | 0.00474 | 0.00517 | 0.00519 |
| All | 32,507 | 0.00462 | 0.00476 | 0.00467 |
Qwen/Qwen3.6-35B-A3B