Qwen3.5-35B-A3B-Base-MNN

MNN HQQ 4-bit inference package converted from Qwen/Qwen3.5-35B-A3B-Base.

Original model

Qwen3.5-35B-A3B-Base is the pretrained-only vision-language checkpoint. Its hybrid architecture combines Gated DeltaNet, gated attention and sparse MoE layers. The upstream card positions this base model for fine-tuning, in-context learning and research; use completion-style prompts for the unmodified checkpoint.

Summary adapted from the upstream model card. Original model capabilities and benchmark claims are separate from the MNN smoke checks below.

Item	Value
Original checkpoint	Qwen/Qwen3.5-35B-A3B-Base
Inputs / outputs in this package	Text and images → text
Layers	40
Routed experts	256
Experts selected per token	8
Text linear quantization	HQQ, 4-bit, group size 64, FP16 scales
Token embedding storage	BF16 (16-bit), separate embedding weights

Download

python -m pip install -U huggingface_hub
python -c "from huggingface_hub import snapshot_download; snapshot_download('taobao-mnn/Qwen3.5-35B-A3B-Base-MNN', local_dir='./Qwen3.5-35B-A3B-Base-MNN')"
export MODEL_DIR="$(pwd)/Qwen3.5-35B-A3B-Base-MNN"

MNN version and build

MNN: 7337404b2815b5fb9ddac13f4ac6a1fb37fcbbb4

This revision includes the MoE export fixes used for this release. Linux CPU build:

git clone https://github.com/alibaba/MNN.git
git -C MNN checkout 7337404b2815b5fb9ddac13f4ac6a1fb37fcbbb4
(cd MNN && ./schema/generate.sh)
cmake -S MNN -B MNN/build -DCMAKE_BUILD_TYPE=Release \
  -DMNN_BUILD_LLM=ON -DMNN_LOW_MEMORY=ON -DMNN_BUILD_CONVERTER=ON \
  -DMNN_SUPPORT_TRANSFORMER_FUSE=ON \
  -DMNN_BUILD_LLM_OMNI=ON -DMNN_BUILD_OPENCV=ON -DMNN_IMGCODECS=ON -DMNN_BUILD_AUDIO=ON
cmake --build MNN/build --parallel 4

Conversion

The quantization arguments match export_args.json. Download the original checkpoint and export it with the revision above:

python -m pip install -r MNN/transformers/llm/export/requirements.txt
python -c "from huggingface_hub import snapshot_download; snapshot_download('Qwen/Qwen3.5-35B-A3B-Base', local_dir='./original/Qwen3.5-35B-A3B-Base', allow_patterns=['*.json', '*.safetensors', '*.model', '*.tiktoken', '*.jinja', '*.txt', '*.py'], ignore_patterns=['original/*', '*.gguf', '*.pt', '*.pth'])"

python MNN/transformers/llm/export/llmexport.py \
  --path ./original/Qwen3.5-35B-A3B-Base --export mnn --hqq \
  --quant_bit 4 --quant_block 64 --scale_bit 16 --embed_bit 16 \
  --lm_quant_bit 4 --lm_quant_block 64 \
  --mnnconvert ./MNN/build/MNNConvert --dst_path ./reexported/Qwen3.5-35B-A3B-Base-MNN

After re-export, reuse this package's verified tokenizer and template. Preserve the newly generated weight offsets and other model configuration:

python - <<'PY'
import json, os, shutil
from pathlib import Path
published = Path(os.environ["MODEL_DIR"])
converted = Path("./reexported/Qwen3.5-35B-A3B-Base-MNN")
for filename in ("llm_config.json", "config.json"):
    path = converted / filename
    config = json.loads(path.read_text())
    reference = json.loads((published / filename).read_text())
    for key in ("jinja", "use_template"):
        config.pop(key, None)
        if key in reference:
            config[key] = reference[key]
    if filename == "config.json":
        config["tokenizer_file"] = "tokenizer.mtok"
    path.write_text(json.dumps(config, ensure_ascii=False, indent=2))
shutil.copyfile(published / "tokenizer.mtok", converted / "tokenizer.mtok")
PY

Run and test

printf 'The capital of France is\n' > prompt.txt
./MNN/build/llm_demo "$MODEL_DIR/config.json" prompt.txt
./MNN/build/llm_bench -m "$MODEL_DIR/config.json"

The original migration passed 10 CPU High/Low smoke cases for this model. These are functional checks, not a broad quality or GPU benchmark.

License and attribution

Weights retain the upstream license and attribution. See the original model card, the license files in this repository and the MNN LLM documentation.

taobao-mnn/Qwen3.5-35B-A3B-Base-MNN