Published by TokForge for on-device inference.

TokForge

• Website: https://tokforge.ai
• Discord: https://discord.gg/Acv3CBtfVm
• Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
• iOS TestFlight: https://testflight.apple.com/join/jnufjzRr

Runs on-device in the TokForge app.

Qwen3-4B Abliterated MNN

MNN 4-bit HQQ quantized model for TokForge on-device inference.

Model Info

Speculative Decoding

This model is compatible with the TokForge Acceleration Pack (Qwen3-0.6B draft model), which works with both censored and abliterated targets.

Usage

Download via TokForge app → Models → Roleplay category, or manually place files in the TokForge models directory.

Limitations and Intended Use

• Intended for TokForge / MNN mobile inference.
• 4B was not the strongest speculative-decoding target in our later preserved fleet results.
• Performance depends strongly on SoC, backend routing, and device thermal behavior.
• This repo is a runtime/export artifact, not a standard Transformers release.

Files

• llm.mnn: Model graph
• llm.mnn.weight: Quantized weights
• tokenizer.txt: Tokenizer vocabulary
• llm_config.json: Model configuration
• embeddings_bf16.bin: Embedding table (8B/14B only)

Credits

• Huihui-Qwen3-4B-abliterated-v2 by huihui-ai (Hugging Face). Used under the Apache-2.0 license.
• Qwen3-4B by the Qwen team (Alibaba Cloud) (Hugging Face). Used under the Apache-2.0 license.
• Converted with llmexport from MNN by Alibaba (Apache-2.0).

Community

• Website: tokforge.ai
• Discord: Join the Discord