Published by TokForge for on-device inference.

TokForge

• Website: https://tokforge.ai
• Discord: https://discord.gg/Acv3CBtfVm
• Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
• iOS TestFlight: https://testflight.apple.com/join/jnufjzRr

Runs on-device in the TokForge app.

Huihui-Qwen3-30B-A3B-abliterated-MNN

Introduction

This model is a 4-bit MNN export of huihui-ai/Qwen3-30B-A3B-abliterated, prepared with the local TokForge llmexport pipeline for on-device and host-side MNN inference.

Source

• Upstream abliterated checkpoint: huihui-ai/Qwen3-30B-A3B-abliterated
• Base model: Qwen/Qwen3-30B-A3B
• Source snapshot SHA: a985d50f8ff3152364f35b814431b32803bd558f

Bundle contents

• config.json
• llm_config.json
• llm.mnn
• llm.mnn.weight
• embeddings_bf16.bin
• tokenizer.txt
• export_args.json
• llm.mnn.json

Quantization

• LLM weights: Q4 HQQ
• Weight block size: 64
• LM head: Q4 block 64
• Embeddings: BF16
• Runtime default: CPU, 4 threads, low precision

Download

pip install huggingface_hub

hf download darkmaniac7/Huihui-Qwen3-30B-A3B-abliterated-MNN --local-dir path/to/model

Usage

git clone https://github.com/alibaba/MNN.git
cd MNN
mkdir build && cd build
cmake .. -DMNN_LOW_MEMORY=true -DMNN_CPU_WEIGHT_DEQUANT_GEMM=true -DMNN_BUILD_LLM=true -DMNN_SUPPORT_TRANSFORMER_FUSE=true
make -j

./llm_demo /path/to/Huihui-Qwen3-30B-A3B-abliterated-MNN/config.json prompt.txt

Notes

• The export was validated locally with the TokForge build-host/llm_demo binary and a 1-token smoke decode.
• This is a safety-reduced / abliterated model. Review usage carefully before publishing or deploying.

Document

MNN-LLM

Credits

• Qwen3-30B-A3B-abliterated by huihui-ai (Hugging Face). Used under the Apache-2.0 license.
• Qwen3-30B-A3B by the Qwen team (Alibaba Cloud) (Hugging Face). Used under the Apache-2.0 license.
• Converted with llmexport from MNN by Alibaba (Apache-2.0).