wemm-embedding-4b for the Ryzen AI NPU

WeMM-Embedding-4B, converted to run on the Ryzen AI NPU of AMD Strix Halo inside halogen-flash-server, beside the Flash model on the GPU and behind the same port. Questions go to the Discord. Bugs go to the server repository's issues.

These files run only inside the halogen-flash-server container, on the NPU. They will not load in transformers or any other runtime.

What is here

vis/                     404 MB  the NPU program and weights for the image encoder, and its NOTICE.md
wemm-embedding-4b.hnpw    5.69 GB  the text model's weights, converted for the NPU
tokenizer/                  20 MB  the tokenizer
LICENSE                            the source model's license, as published

Its text runs on the NPU program of halogen-npu-decider-4b, so the server fetches that repository's devices/ too, and a host that serves both fetches it once. Its images go through its own image encoder, in vis/.

Use it

Start halogen-flash-server 0.18.3 or later with HALOGEN_DOWNLOAD on and HALOGEN_NPU_MODELS=wemm-embedding-4b. The server fetches these files into /models/npu/wemm-embedding-4b/ and checks each one's size and checksum before it starts. It answers /v1/embeddings with 2,560-dimension vectors (fewer with dimensions), in the OpenAI request shape, for texts up to 4,096 tokens and for images. Texts and images share one vector space, so a text can find an image and the other way round, and one request may mix them.

The files at the head of this repository need halogen-flash-server 0.18.12 or later; each server version downloads the revision it was released with, so an older server keeps working unchanged.

An image is an input item of the form {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}: PNG, JPEG, WebP, GIF or BMP, sent inline (the server does not fetch URLs). The image encoder takes up to 768 x 768 pixels: a larger image is scaled down to fit, its aspect kept, and every image is sized to a multiple of 32 pixels a side. An image no larger in area than 512 x 512 pixels runs on the encoder's smaller program.

The host needs the NPU driver, XRT with its NPU plugin, and the GPU's fabric clock held while the NPU works beside the GPU. The setup and every request shape are in docs/NPU.md.

Source and license

Converted from tencent/WeMM-Embedding-4B at revision 41a4fafdd26252ddff05c025413af7e61e26f08e. Apache-2.0, as the source. The source builds on Qwen3.5-4B (Copyright 2026 Alibaba Cloud, Apache-2.0). LICENSE is the source's license file, unchanged. The NPU programs' third-party notices are in vis/NOTICE.md here and in devices/NOTICE.md of halogen-npu-decider-4b.