d3-nano for the Ryzen AI NPU

vllm-sr/d3-nano, the 2B decision model of vLLM Semantic Router's Decision 3.0, converted to run on the Ryzen AI NPU of AMD Strix Halo inside halogen-flash-server, beside the Flash model on the GPU and behind the same port. Questions go to the Discord. Bugs go to the server repository's issues.

These files run only inside the halogen-flash-server container, on the NPU. They will not load in transformers or any other runtime.

What is here

devices/             211 MB  the NPU program for this model, and its NOTICE.md
d3-nano.hnpw        2.59 GB  the weights, converted for the NPU
vis/                 407 MB  the NPU program and weights of the model's vision tower, and its NOTICE.md
tokenizer/            13 MB  the model's own tokenizer

Use it

Start halogen-flash-server 0.18.11 or later with HALOGEN_DOWNLOAD on and HALOGEN_NPU_MODELS=d3-nano. The server fetches these files into /models/npu/d3-nano/ and checks each one's size and checksum before it starts. A request's model may name d3-nano or the upstream name vllm-sr/d3-nano.

The files at the head of this repository need halogen-flash-server 0.18.12 or later; each server version downloads the revision it was released with, so an older server keeps working unchanged.

It answers System One questions on POST /v1/systemone exactly as d3-lite does (a choice of 1 to 255 options, a noul, a score), with better answers on harder questions: on our decision benchmark it scores about 0.73 / 0.67 at 256 / 1,024 tokens against d3-lite's 0.56 / 0.59. The answers match the model's own runtime on the CPU: the NPU picked the same answer on 502 of 512 questions, and those it did not were near ties. Prompts go up to 8,192 tokens.

Pictures: a request may carry one picture in images (a list of one: a data: URL or base64, as a string or as {"url": ...}), and every question in the request sees it. The server sizes it as the model's own processor does, within its pixel budget (65,536 to 1,638,400 pixels), with the top lowered to 768 x 768 or the same area at another shape (2,304 patches of 16 pixels), the most the NPU's vision tower takes. Video is not served. On 64 CV-Bench questions the NPU picked the model's own CPU answer on all 64.

On one Strix Halo, one question served alone takes about 105 ms at a short state, 300 ms at 700 tokens and 1.1 s at 3,600 tokens (the request's wall, HTTP included): 1.2 to 1.7 times d3-lite. A question about a picture takes about 0.35 s at 256 x 256 and 0.87 s at 768 x 768, the vision tower included.

The host needs the NPU driver, XRT with its NPU plugin, and the GPU's fabric clock held while the NPU works beside the GPU. The setup and every request shape are in docs/NPU.md.

Source and license

Converted from vllm-sr/d3-nano at revision 462efc366229ea268cf4eed3738b3fc4415e4817, itself built on Qwen/Qwen3.5-2B. Apache-2.0, as the source. devices/NOTICE.md lists the third-party notices the NPU program carries.