Kindi MTP heads
Draft heads for Kindi, the fast inference engine for low-cost GPUs. A head makes the model write faster. The output does not change.
Kindi downloads these files for you. You do not need to download them yourself.
What does a head do?
A head is a small extra part of the model. It guesses the tokens that come after the next token. The main model then checks all the guesses in one pass, and Kindi keeps only the guesses that agree with the main model. Thus the text is the same as without the head. Only the speed changes.
How fast is it?
Kindi 0.4.0, greedy decoding, the same card and the same text, tokens per second:
| Card and context | Without the head | With the head | Gain |
|---|---|---|---|
| RTX 3060, 512 tokens | 316.9 | 350.7 | +11% |
| RTX 3060, 8K | 314.4 | 340.3 | +8% |
| RTX 3080 (190 W), 512 tokens | 370.8 | 409.0 | +10% |
| RTX 3080 (190 W), 8K | 368.6 | 395.5 | +7% |
Kindi uses the head on NVIDIA cards with tensor cores (RTX 30 and newer), up to 8K tokens of context. Above 8K, Kindi turns the head off, because it does not make the model faster there.
Files
| File | Size | sha256 | For |
|---|---|---|---|
kindi-mtp-2b-v1.npz |
60,960,552 bytes | 1b5c0c790a594af368c3646370c527c319cdb3bb0592dd3e523e752f721a5520 |
Ternary-Bonsai-2-2B |
Only Kindi reads this file format.
How do I use it?
Install Kindi from amirhusain.com/kindi. The installer downloads the head and checks its sha256. To set it by hand:
KINDI_SPEC=auto KINDI_MTP_HEAD=/path/to/kindi-mtp-2b-v1.npz
Credits
- Base model: Ternary-Bonsai-2-2B by Prism ML (Apache-2.0). This head is our own work, made for that model.
- This head contains weights derived from Qwen3.5-2B by Alibaba Cloud (Apache-2.0).
License
Copyright 2026 Amir Husain. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Qwen3.5-2B: Copyright Alibaba Cloud, Apache-2.0. Ternary-Bonsai-2-2B: Copyright Prism ML, Apache-2.0.