qwen3guard-gen-0.6b for the Ryzen AI NPU

Qwen3Guard-Gen-0.6B, converted to run on the Ryzen AI NPU of AMD Strix Halo inside halogen-flash-server, beside the Flash model on the GPU and behind the same port. Questions go to the Discord. Bugs go to the server repository's issues.

These files run only inside the halogen-flash-server container, on the NPU. They will not load in transformers or any other runtime.

What is here

qwen3guard-gen-0.6b.hnpw    814 MB  the weights, converted for the NPU
tokenizer/            11 MB  the tokenizer

This model runs on the NPU program of halogen-npu-qwen3-embedding-0.6b, which the server fetches with it. It takes up to 4,096 tokens a prompt, its template included.

Use it

Start halogen-flash-server 0.16.2 or later with HALOGEN_DOWNLOAD on and HALOGEN_NPU_MODELS=qwen3guard-gen-0.6b. The server fetches these files into /models/npu/qwen3guard-gen-0.6b/ and checks each one's size and checksum before it starts. It answers OpenAI's /v1/moderations: flagged is the model's Unsafe, "strict": true also flags Controversial, and each result adds label and label_scores.

The host needs the NPU driver, XRT with its NPU plugin, and the GPU's fabric clock held while the NPU works beside the GPU. The setup, every request shape, and how to run your own fine-tune of this model are in docs/NPU.md.

Source and license

Converted from Qwen/Qwen3Guard-Gen-0.6B at revision fada3b2f655b89601929198343c94cd2f64d93cc. Apache-2.0, as the source. The NPU program's third-party notices are in the embedding repository's devices/NOTICE.md.