This model is an OfflineModel of Qwen3.5-0.8B converted using the Cunxin 3.0 toolchain, quantized with W4A16, with a context length of 1024.
The model directory also contains vision_model.ortm, the visual tower of Qwen3.5, so the same directory can be used for image question answering.
This OfflineModel supports all chip platforms corresponding to Cunxin 3.0. Please refer to the npu_toolchain_doc for details.
Option 1: Download the pre-converted OfflineModel directly from ModelScope or HuggingFace
# ModelScope
modelscope download 'oritek/Qwen3.5-0.8B-Cunxin_3_0-W4A16-CTX1024' \
--local-dir './Qwen3.5-0.8B-Cunxin_3_0-W4A16-CTX1024'
# HuggingFace (consistent with script defaults: mirror endpoint + Xet disabled)
export HF_ENDPOINT="${HF_ENDPOINT:-https://hf-mirror.com}" HF_HUB_DISABLE_XET=1
hf download 'oritek/Qwen3.5-0.8B-Cunxin_3_0-W4A16-CTX1024' \
--local-dir './Qwen3.5-0.8B-Cunxin_3_0-W4A16-CTX1024'
If your driver is not the latest version, download the matching release branch (branch name can be found on the repository page; --revision also accepts tags/commits) instead of the latest artifact on the default branch.
Option 2: Convert using omg
omg --model Qwen/Qwen3.5-0.8B/model.safetensors.index.json \
--hf_config Qwen/Qwen3.5-0.8B/config.json \
--tokenizer Qwen/Qwen3.5-0.8B/tokenizer.json \
--config input_cfg.yml \
--output_model ./Qwen3.5-0.8B-Cunxin_3_0-W4A16-CTX1024 \
--target XXX_TARGET
openai_server supports two modes:
./openai_server \
--model ./Qwen3.5-0.8B-Cunxin_3_0-W4A16-CTX1024 \
--prompt "Who are you?"
./openai_server \
--model ./Qwen3.5-0.8B-Cunxin_3_0-W4A16-CTX1024 \
--host 0.0.0.0 --port 8000
此模型为 Qwen3.5-0.8B 在 Cunxin 3.0 工具链转换的 OfflineModel,采用 W4A16 量化方式,上下文长度 1024。
模型目录内另含 vision_model.ortm(Qwen3.5 的视觉塔),因此同一目录也可用于图片问答。
此 OfflineModel 支持 Cunxin 3.0 对应的所有芯片平台,具体参考《npu_toolchain_doc》。
方法 1:直接从 ModelScope 或 HuggingFace 上下载转换好的 OfflineModel
# ModelScope
modelscope download 'oritek/Qwen3.5-0.8B-Cunxin_3_0-W4A16-CTX1024' \
--local-dir './Qwen3.5-0.8B-Cunxin_3_0-W4A16-CTX1024'
# HuggingFace(与脚本默认一致:镜像端点 + 关闭 Xet)
export HF_ENDPOINT="${HF_ENDPOINT:-https://hf-mirror.com}" HF_HUB_DISABLE_XET=1
hf download 'oritek/Qwen3.5-0.8B-Cunxin_3_0-W4A16-CTX1024' \
--local-dir './Qwen3.5-0.8B-Cunxin_3_0-W4A16-CTX1024'
驱动不是最新版本时,下载与之配套的发布分支(分支名见仓库页面,--revision 也接受 tag / commit),而不是默认分支上的最新产物。
方法 2:使用 omg 转换
omg --model Qwen/Qwen3.5-0.8B/model.safetensors.index.json \
--hf_config Qwen/Qwen3.5-0.8B/config.json \
--tokenizer Qwen/Qwen3.5-0.8B/tokenizer.json \
--config input_cfg.yml \
--output_model ./Qwen3.5-0.8B-Cunxin_3_0-W4A16-CTX1024 \
--target XXX_TARGET
openai_server 有两种使用模式:
./openai_server \
--model ./Qwen3.5-0.8B-Cunxin_3_0-W4A16-CTX1024 \
--prompt "Who are you?"
./openai_server \
--model ./Qwen3.5-0.8B-Cunxin_3_0-W4A16-CTX1024 \
--host 0.0.0.0 --port 8000
Qwen/Qwen3.5-0.8B