This model is an OfflineModel of Qwen3-1.7B converted using the Cunxin 3.0 toolchain, quantized with W4A16, with a context length of 1024.
This OfflineModel supports all chip platforms corresponding to Cunxin 3.0. Please refer to the npu_toolchain_doc for details.
Option 1: Download the pre-converted OfflineModel directly from ModelScope or HuggingFace
# ModelScope
modelscope download 'oritek/Qwen3-1.7B-Cunxin_3_0-W4A16-CTX1024' \
--local-dir './Qwen3-1.7B-Cunxin_3_0-W4A16-CTX1024'
# HuggingFace (consistent with script defaults: mirror endpoint + Xet disabled)
export HF_ENDPOINT="${HF_ENDPOINT:-https://hf-mirror.com}" HF_HUB_DISABLE_XET=1
hf download 'oritek/Qwen3-1.7B-Cunxin_3_0-W4A16-CTX1024' \
--local-dir './Qwen3-1.7B-Cunxin_3_0-W4A16-CTX1024'
If your driver is not the latest version, download the matching release branch (branch name can be found on the repository page; --revision also accepts tags/commits) instead of the latest artifact on the default branch.
Option 2: Convert using omg
omg --model Qwen/Qwen3-1.7B/model.safetensors.index.json \
--hf_config Qwen/Qwen3-1.7B/config.json \
--tokenizer Qwen/Qwen3-1.7B/tokenizer.json \
--config input_cfg.yml \
--output_model ./Qwen3-1.7B-Cunxin_3_0-W4A16-CTX1024 \
--target XXX_TARGET
openai_server supports two modes:
./openai_server \
--model ./Qwen3-1.7B-Cunxin_3_0-W4A16-CTX1024 \
--prompt "Who are you?"
./openai_server \
--model ./Qwen3-1.7B-Cunxin_3_0-W4A16-CTX1024 \
--host 0.0.0.0 --port 8000
此模型为 Qwen3-1.7B 在 Cunxin 3.0 工具链转换的 OfflineModel,采用 W4A16 量化方式,上下文长度 1024。
此 OfflineModel 支持 Cunxin 3.0 对应的所有芯片平台,具体参考《npu_toolchain_doc》。
方法 1:直接从 ModelScope 或 HuggingFace 上下载转换好的 OfflineModel
# ModelScope
modelscope download 'oritek/Qwen3-1.7B-Cunxin_3_0-W4A16-CTX1024' \
--local-dir './Qwen3-1.7B-Cunxin_3_0-W4A16-CTX1024'
# HuggingFace(与脚本默认一致:镜像端点 + 关闭 Xet)
export HF_ENDPOINT="${HF_ENDPOINT:-https://hf-mirror.com}" HF_HUB_DISABLE_XET=1
hf download 'oritek/Qwen3-1.7B-Cunxin_3_0-W4A16-CTX1024' \
--local-dir './Qwen3-1.7B-Cunxin_3_0-W4A16-CTX1024'
驱动不是最新版本时,下载与之配套的发布分支(分支名见仓库页面,--revision 也接受 tag / commit),而不是默认分支上的最新产物。
方法 2:使用 omg 转换
omg --model Qwen/Qwen3-1.7B/model.safetensors.index.json \
--hf_config Qwen/Qwen3-1.7B/config.json \
--tokenizer Qwen/Qwen3-1.7B/tokenizer.json \
--config input_cfg.yml \
--output_model ./Qwen3-1.7B-Cunxin_3_0-W4A16-CTX1024 \
--target XXX_TARGET
openai_server 有两种使用模式:
./openai_server \
--model ./Qwen3-1.7B-Cunxin_3_0-W4A16-CTX1024 \
--prompt "Who are you?"
./openai_server \
--model ./Qwen3-1.7B-Cunxin_3_0-W4A16-CTX1024 \
--host 0.0.0.0 --port 8000
Qwen/Qwen3-1.7B