qwen3guard-stream-0.6b for the Ryzen AI NPU

Qwen3Guard-Stream-0.6B, converted to run on the Ryzen AI NPU of AMD Strix Halo inside halogen-flash-server, beside the Flash model on the GPU and behind the same port. Questions go to the Discord. Bugs go to the server repository's issues.

These files run only inside the halogen-flash-server container, on the NPU. They will not load in transformers or any other runtime.

What is here

qwen3guard-stream-0.6b.hnpw    the weights, converted for the NPU
heads.npz                      the model's safety heads
tokenizer/                     the tokenizer

This model runs on the NPU program of halogen-npu-qwen3-embedding-0.6b, which the server fetches with it.

Use it

Start halogen-flash-server 0.18.3 or later with HALOGEN_DOWNLOAD on and HALOGEN_NPU_MODELS=qwen3guard-stream-0.6b. The server fetches these files into /models/npu/qwen3guard-stream-0.6b/ and checks each one's size and checksum before it starts.

It watches a chat completion's reply as it is written, when the request asks for it with a guard field or the X-Halogen-Guard header. The reply itself is never held back or changed. Its stream carries the guard's verdicts beside the text, and a summary at the end counts the flagged parts and names the first. A part is flagged when the model's Unsafe probability reaches 0.8 (HALOGEN_GUARD_FLAG, or flag in the request's guard object). To moderate a whole prompt or reply in one call instead, use qwen3guard-gen-0.6b on /v1/moderations.

The host needs the NPU driver, XRT with its NPU plugin, and the GPU's fabric clock held while the NPU works beside the GPU. The setup and every request shape are in docs/NPU.md.

Source and license

Converted from Qwen/Qwen3Guard-Stream-0.6B at revision 74e1479150e9029d6778993f00491108323bb6f8. Apache-2.0, as the source.