Maple-Preview-GGUFs
Custom llama.cpp fork and setup instructions: github.com/deepgrove-ai/llama.cpp
We include the following gguf variants:
| Variant | GGUF size |
|---|---|
| TQ1_0 + Q4_K head | 4.64 GiB |
| TQ1_0 + FP16 head | 5.06 GiB |
| TQ2_0 + Q4_K head | 5.50 GiB |
| TQ2_0 + FP16 head | 5.91 GiB |
TQ1_0 and TQ2_0 are different ternary packing schemes. Use TQ2_0 for generally faster speeds but slightly higher memory. LM-head is kept in higher precision - either Q4_k or FP16.
Speed
M5 Pro, CPU-only, 16 threads, 512 prompt tokens, 128 generated tokens, 3 repetitions:
| Matrix weights | LM head | GGUF size | Prefill (pp512) | Decode (tg128) |
|---|---|---|---|---|
| TQ1_0 | FP16 | 5.06 GiB | 515.41 ± 0.28 tokens/s | 161.06 ± 0.57 tokens/s |
| TQ1_0 | Q4_K | 4.64 GiB | 513.33 ± 3.62 tokens/s | 231.13 ± 0.13 tokens/s |
| TQ2_0 | FP16 | 5.91 GiB | 618.57 ± 2.12 tokens/s | 169.81 ± 2.94 tokens/s |
| TQ2_0 | Q4_K | 5.50 GiB | 610.48 ± 3.76 tokens/s | 252.74 ± 0.37 tokens/s |
Architecture
Maple-Preview is a 20B-A1B reasoning model designed from the start for efficient on-device inference. It utilizes a 24-layer, 256-expert (8 active) configuration with 3:1 SWA-512:GA attention.
Evaluation
On benchmarks, Maple-Preview sets a new point on the Pareto frontier for both memory-to-performance and speed-to-performance, demonstrating its strong reasoning capabilities. However, we note that this preview is focused primarily on raw reasoning and, as such, may underperform on agentic benchmarks. We intend to continue improving general performance through extended training before Maple's full release.
License
Maple-Preview is released under the MIT License.