Published by TokForge for on-device inference.

TokForge

Website: https://tokforge.ai
Discord: https://discord.gg/Acv3CBtfVm
Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
iOS TestFlight: https://testflight.apple.com/join/jnufjzRr

Runs on-device in the TokForge app.

Huihui-Qwen3.6-27B-abliterated-MNN

MNN-format 4-bit TokForge export of huihui-ai/Huihui-Qwen3.6-27B-abliterated, packaged for Android on-device inference with TokForge.

What this is

Source checkpoint: huihui-ai/Huihui-Qwen3.6-27B-abliterated
Base model: Qwen/Qwen3.6-27B
Source snapshot SHA: 27502c8717fd5a2f8c0c77188c10c243fd4f672e
Architecture: qwen3_5, 64 layers, hidden size 5120, 24 attention heads, 4 KV heads, hybrid linear + full attention, MRoPE, DeepStack vision sidecar.
Target runtime: TokForge Android MNN runtime. Join the TokForge community on Discord.

The bundle includes the visual MNN sidecar, but TokForge's device checks covered text generation only.

Bundle contents

config.json - TokForge/MNN runtime defaults.
llm_config.json - model geometry and tokenizer/chat settings.
llm.mnn / llm.mnn.weight - quantized LLM graph and external weights.
embeddings_bf16.bin - separated BF16 embedding table.
tokenizer.txt - tokenizer exported by the MNN pipeline.
visual.mnn / visual.mnn.weight - Qwen3.6 vision sidecar.
export_args.json - conversion settings.
llm.mnn.json - exported graph metadata.
tokforge_benchmarks.json - raw summary of TokForge's device checks.

Quantization scheme

Flag	Value
`--quant_bit`	4
`--quant_block`	64
`--lm_quant_bit`	4
`--lm_quant_block`	64
`--embed_bit`	16
`--hqq`	enabled
`--seperate_embed`	enabled

Runtime defaults in config.json: CPU backend, 4 threads, low precision, low memory, separated BF16 embeddings.

Device notes

In TokForge's device checks (April 30 and May 1, 2026, four test prompts per phone), the 24 GB devices completed all four prompts and are practical for short local text generation with this export. The 16 GB devices can load the model, but longer paragraph/code prompts are not practical in this CPU MNN configuration: OnePlus and Lenovo completed short/code prompts but timed out on paragraph generation; S26 and Pixel completed only arithmetic and short sentence before timing out on longer prompts.

Download

pip install -U huggingface_hub
hf download darkmaniac7/Huihui-Qwen3.6-27B-abliterated-MNN --local-dir Huihui-Qwen3.6-27B-abliterated-MNN

Usage with upstream MNN `llm_demo`

git clone https://github.com/alibaba/MNN.git
cd MNN
mkdir build && cd build
cmake .. -DMNN_LOW_MEMORY=true -DMNN_CPU_WEIGHT_DEQUANT_GEMM=true \
         -DMNN_BUILD_LLM=true -DMNN_SUPPORT_TRANSFORMER_FUSE=true
make -j

./llm_demo /path/to/Huihui-Qwen3.6-27B-abliterated-MNN/config.json prompt.txt

Host smoke validation with the TokForge MNN build loaded successfully, detected 48 linear-attention state layers out of 64 total layers, and emitted the expected one-token OK response for a no-thinking prompt.

Credits

Huihui-Qwen3.6-27B-abliterated by huihui-ai (Hugging Face). Used under the Apache-2.0 license.
Qwen3.6-27B by the Qwen team (Alibaba Cloud) (Hugging Face). Used under the Apache-2.0 license.
Converted with llmexport from MNN by Alibaba (Apache-2.0).

License and safety

Apache-2.0, inherited from the upstream Qwen and huihui-ai model cards. This is a safety-reduced / abliterated model; deploy with appropriate product policy, user controls, and local-law awareness.

darkmaniac7/Huihui-Qwen3.6-27B-abliterated-MNN