Published by TokForge for on-device inference.
TokForge
- Website: https://tokforge.ai
- Discord: https://discord.gg/Acv3CBtfVm
- Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
- iOS TestFlight: https://testflight.apple.com/join/jnufjzRr
Runs on-device in the TokForge app.
Qwen3-1.7B-MNN-HQQ
Qwen3 1.7B re-exported to MNN with HQQ int4 quantization for on-device inference with TokForge.
Original model by the Qwen team. This repository only changes the runtime format and quantization for mobile deployment.
What is different
Compared with the MNN export TokForge used before (taobao-mnn/Qwen3-1.7B-MNN):
- Re-exported with HQQ int4: asymmetric HQQ 4-bit weights, block size 128, int4 output head (tied with the input embeddings).
- Lower KL divergence against the BF16 source: mean KL 0.33 vs 0.57, top-1 token agreement 79.9% vs 75.8%. Measured on a CPU host over 64 prompts x 128 positions on MNN's low-memory route.
- Fewer errors in long multi-turn chats on TokForge 1.3.7 and later. Measured on graded 10-turn and 12-turn chats past 2,048 tokens, on phones.
- Smaller download: 0.97 GB of runtime files.
- Unchanged: tokenizer and chat template.
Model Details
| Architecture | Qwen3 dense, 28 layers, GQA 16Q/8KV, hidden size 2048 |
| Parameters | 1.7B (4-bit quantized) |
| Format | MNN (Alibaba Mobile Neural Network) |
| Quantization | HQQ asymmetric int4, block 128, int4 tied head and embeddings |
| Vocab | 151,936 tokens |
| Source | Qwen/Qwen3-1.7B at 70d244cc86ccca08cf5af4e1e306ecf908b1ad5e |
| Recommended app | TokForge 1.3.7 or later |
Files
| File | Description |
|---|---|
llm.mnn |
Model computation graph |
llm.mnn.weight |
Quantized weight data (HQQ int4, block 128) |
llm_config.json |
Model config with the Jinja chat template |
tokenizer.txt |
Tokenizer vocabulary |
config.json |
MNN runtime config |
LICENSE |
Apache-2.0 license text from the source repository |
SOURCE_MODEL_CARD.md |
The original Qwen3-1.7B model card, unchanged |
ATTRIBUTION.txt |
Source attribution |
MODIFICATIONS.txt |
What this conversion changed, with file hashes |
Usage with TokForge
- Install TokForge (version 1.3.7 or later recommended).
- Open the app, go to Models and download Qwen3 1.7B.
- Start chatting. Everything runs locally on the phone.
Recommended Settings
TokForge picks these automatically.
| Setting | Value |
|---|---|
| Backend | CPU |
| Precision | Low |
| Threads | 2 to 4 |
| Thinking | Off |
Attribution
This is an MNN conversion of Qwen3-1.7B by the
Qwen team, released under the Apache License 2.0. All credit for the model
architecture and training goes to the original authors. The source training weights were not edited. The license text
and the original model card are included unchanged, and MODIFICATIONS.txt lists what the conversion changed.
Limitations
- Intended for TokForge and MNN on-device inference. This is a runtime bundle, not a Transformers training checkpoint.
- 4-bit quantization can still reduce quality compared with the full-precision original.
- Text only. The source model has no vision or audio components.
- A small model: it can still get facts wrong. Check important answers.
Export Details
Converted with MNN's llmexport and MNNConvert from TokForge's MNN fork at commit
e0f197d0c691874e0ca21398eaba309969cdaf32 (the engine in TokForge 1.3.6.1):
python llmexport.py --path Qwen/Qwen3-1.7B --export mnn \
--quant_bit 4 --quant_block 128 --lm_quant_bit 4 --lm_quant_block 128 --hqq --embed_bit 16
Source revision: 70d244cc86ccca08cf5af4e1e306ecf908b1ad5e.
The runtime files keep the tokenizer bytes and tied-embedding layout of the previous TokForge download.
Credits
- Qwen3-1.7B by the Qwen team (Alibaba Cloud) (Hugging Face). Used under the Apache-2.0 license.
- Converted with llmexport from MNN by Alibaba (Apache-2.0).
Community
- Website: tokforge.ai
- Discord: Join our Discord