Not recommended. This is an experimental build. Use magi-1.0 instead.
Paper: Where the Bits Go (Sun, 2026) · Recipe and data: GEHIRNS/magi · Website: gehirns.pages.dev
Introduction
An alternative to magi-1.0 that spends extra bits on attention. It is published so the comparison can be reproduced. For use, take magi-1.0, which is smaller and scored slightly higher.
magi-beta-1 is Qwen/Qwen3.5-4B in a mixture of standard GGUF types. It differs from magi-1.0 in one place: the key, value and output projections of the 8 full-attention layers are stored at Q6_K instead of Q4_K. These are the most sensitive tensors in our measurements, yet the extra 0.030 GiB did not buy measurable accuracy (−0.65 points against magi-1.0, paired 95% interval −1.72 to +0.44). That supports the main finding: magi-1.0 already protects what matters. It involves no training and runs unmodified in llama.cpp and Ollama.
- 3.712 bits per weight all-in: 1.817 GiB of tensor data for 4,205,751,296 parameters (1.83 GiB file)
- Text only: the vision projector of Qwen3.5-4B is not included.
Benchmarks
Same protocol as magi-1.0: one RTX 5070 (12 GB), 12 suites and 2,313 tasks, thinking on with a 4,096-token budget, greedy decoding, 32K context. Small numbers in the Ours columns are relative to BF16.
| Baselines | Ours | ||||
|---|---|---|---|---|---|
| Benchmark | BF16 8.98 GiB | Q4_K_M 3.78 GiB | Q2_K 3.15 GiB | magi-1.0 3.02 GiB | magi-beta-1 3.07 GiB |
| Efficiency | |||||
| GPU memory at 32K (GiB) | 8.98 | 3.78 | 3.15 | 3.023.0× less | 3.072.9× less |
| Benchmark score | 83.96% | 84.14% | 77.06% | 81.30%97% | 80.65%96% |
| Capability density | 0.294 | 0.703 | 0.675 | 0.8002.7× | 0.7712.6× |
| Decode speed (tok/s) | 66.3 | 146.6 | 165.5 | 180.02.7× | 181.02.7× |
| Traces ending in a loop | 3.4% | 3.3% | 33.3% | 9.4% | 8.5% |
| Coding | |||||
| HumanEval+ | 91.46 | 92.68 | 75.00 | 87.80 | 87.20 |
| MBPP+ | 72.18 | 73.68 | 58.90 | 68.42 | 70.43 |
| LiveCodeBench | 35.33 | 35.33 | 15.33 | 24.67 | 20.00 |
| Math | |||||
| GSM8K | 87.33 | 93.33 | 90.67 | 89.33 | 88.67 |
| Science and Knowledge | |||||
| ARC-Challenge | 98.67 | 98.67 | 97.33 | 97.33 | 98.00 |
| MMLU | 84.50 | 82.50 | 74.00 | 82.00 | 81.50 |
| OpenBookQA | 95.00 | 96.50 | 93.00 | 96.00 | 95.50 |
| SciQ | 98.00 | 98.50 | 96.00 | 96.00 | 98.00 |
| TruthfulQA | 82.50 | 82.00 | 73.00 | 79.50 | 77.50 |
| Commonsense | |||||
| HellaSwag | 75.50 | 72.00 | 71.50 | 72.00 | 68.00 |
| WinoGrande | 87.00 | 84.50 | 80.00 | 82.50 | 83.00 |
| Tool Use | |||||
| Function calling | 100.00 | 100.00 | 100.00 | 100.00 | 100.00 |
Recipe
| Tensor group | Type |
|---|---|
| FFN gate, up, down (all 32 layers); gated-attention output gate (24) | IQ3_XXS |
DeltaNet attn_qkv, DeltaNet ssm_out, full-attention q, token embedding = tied head | Q4_K |
| Full-attention k, v, output (8 layers) | Q6_K |
DeltaNet gates ssm_alpha, ssm_beta | Q8_0 |
| Norms, convolution, A, Δt bias | F32 |
How to run
llama-server -m magi-beta-1.gguf -c 32768 --jinja
# or, with Ollama (0.40.3 verified to use the embedded template):
ollama create magi-beta-1 -f Modelfile && ollama run magi-beta-1
The 8-bit KV cache was not benchmarked for this model.
Limitations
The limitations of magi-1.0 apply. The model was evaluated on one benchmark, one runtime and one GPU, once per configuration. LiveCodeBench is the weakest suite (20.0% vs BF16's 35.3%). Text only. The name is ours, and this is not a fine-tune.
License
Apache 2.0, inherited from Qwen/Qwen3.5-4B. The license text is included in this repository.