Published by TokForge for on-device inference.
TokForge
- Website: https://tokforge.ai
- Discord: https://discord.gg/Acv3CBtfVm
- Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
- iOS TestFlight: https://testflight.apple.com/join/jnufjzRr
Runs on-device in the TokForge app.
Huihui-Qwen3.6-27B-abliterated-MNN
MNN-format 4-bit TokForge export of huihui-ai/Huihui-Qwen3.6-27B-abliterated, packaged for Android on-device inference with TokForge.
What this is
- Source checkpoint:
huihui-ai/Huihui-Qwen3.6-27B-abliterated - Base model:
Qwen/Qwen3.6-27B - Source snapshot SHA:
27502c8717fd5a2f8c0c77188c10c243fd4f672e - Architecture:
qwen3_5, 64 layers, hidden size 5120, 24 attention heads, 4 KV heads, hybrid linear + full attention, MRoPE, DeepStack vision sidecar. - Target runtime: TokForge Android MNN runtime. Join the TokForge community on Discord.
The bundle includes the visual MNN sidecar, but TokForge's device checks covered text generation only.
Bundle contents
config.json- TokForge/MNN runtime defaults.llm_config.json- model geometry and tokenizer/chat settings.llm.mnn/llm.mnn.weight- quantized LLM graph and external weights.embeddings_bf16.bin- separated BF16 embedding table.tokenizer.txt- tokenizer exported by the MNN pipeline.visual.mnn/visual.mnn.weight- Qwen3.6 vision sidecar.export_args.json- conversion settings.llm.mnn.json- exported graph metadata.tokforge_benchmarks.json- raw summary of TokForge's device checks.
Quantization scheme
| Flag | Value |
|---|---|
--quant_bit |
4 |
--quant_block |
64 |
--lm_quant_bit |
4 |
--lm_quant_block |
64 |
--embed_bit |
16 |
--hqq |
enabled |
--seperate_embed |
enabled |
Runtime defaults in config.json: CPU backend, 4 threads, low precision, low memory, separated BF16 embeddings.
Device notes
In TokForge's device checks (April 30 and May 1, 2026, four test prompts per phone), the 24 GB devices completed all four prompts and are practical for short local text generation with this export. The 16 GB devices can load the model, but longer paragraph/code prompts are not practical in this CPU MNN configuration: OnePlus and Lenovo completed short/code prompts but timed out on paragraph generation; S26 and Pixel completed only arithmetic and short sentence before timing out on longer prompts.
Download
pip install -U huggingface_hub
hf download darkmaniac7/Huihui-Qwen3.6-27B-abliterated-MNN --local-dir Huihui-Qwen3.6-27B-abliterated-MNN
Usage with upstream MNN llm_demo
git clone https://github.com/alibaba/MNN.git
cd MNN
mkdir build && cd build
cmake .. -DMNN_LOW_MEMORY=true -DMNN_CPU_WEIGHT_DEQUANT_GEMM=true \
-DMNN_BUILD_LLM=true -DMNN_SUPPORT_TRANSFORMER_FUSE=true
make -j
./llm_demo /path/to/Huihui-Qwen3.6-27B-abliterated-MNN/config.json prompt.txt
Host smoke validation with the TokForge MNN build loaded successfully, detected 48 linear-attention state layers out of 64 total layers, and emitted the expected one-token OK response for a no-thinking prompt.
Credits
- Huihui-Qwen3.6-27B-abliterated by huihui-ai (Hugging Face). Used under the Apache-2.0 license.
- Qwen3.6-27B by the Qwen team (Alibaba Cloud) (Hugging Face). Used under the Apache-2.0 license.
- Converted with llmexport from MNN by Alibaba (Apache-2.0).
License and safety
Apache-2.0, inherited from the upstream Qwen and huihui-ai model cards. This is a safety-reduced / abliterated model; deploy with appropriate product policy, user controls, and local-law awareness.