This version is primarily assembled for Hammy, a
Chinese-language video organizing & archiving tool. The 448-ft model (448-ft/) is
included for Hammy only: it is calibrated for character/IP tagging, the other categories were not tuned for it.
Community conversion of pixai-labs/pixai-tagger-v1.0 (Apache-2.0) to ONNX in FP16 and FP32, plus a script that builds TensorRT engines from them on any GPU.
For the full model documentation (architecture, training data, original usage), see the upstream model card.
Forward time (median, per image; batch 1 unless noted) — RTX 5070 (12 GB), Windows, TensorRT 11.3 / onnxruntime-gpu 1.30:
| runtime | @1008 zs | @672 zs | @672-ft | @336-ft (batch 2) |
|---|---|---|---|---|
| PyTorch FP32 (original checkpoint) | 398 ms | 168 ms | — | — |
| onnxruntime FP16 | 161 ms | 65 ms | 63 ms | 16.1 ms |
| onnxruntime FP32 | — | — | 109 ms | 25.5 ms |
| TensorRT FP16 engine | 61 ms | 26 ms | 25 ms | 7.0 ms |
| TensorRT FP32 engine (query-chunked) | 257 ms | 99 ms | 99 ms | 22.1 ms |
--fp32 on those cards; the query-chunked FP32 engine
also fits their 6 GB of VRAM.| file | description |
|---|---|
tagger_1008_fp16.onnx / tagger_672_fp16.onnx |
zero-shot FP16 ONNX, static (1,3,S,S) → (1,30877), weights inlined (~950 MB each) |
tagger_1008_fp32.onnx / tagger_672_fp32.onnx (+ .onnx.data) |
zero-shot FP32 ONNX, dynamo-exported with query-chunked global attention (fits 6 GB GPUs), ~1.9 GB each |
336-ft/onnx/fp16.onnx / fp32.onnx (+ fp32.onnx.data) |
fine-tuned 336, static (2,3,336,336) → (2,30877) |
672-ft/onnx/fp16.onnx / fp32.onnx (+ fp32.onnx.data) |
fine-tuned 672, static (1,3,672,672) → (1,30877) |
448-ft/onnx/fp16.onnx / fp32.onnx (+ fp32.onnx.data) |
fine-tuned 448 (Hammy only), static (1,3,448,448) → (1,30877) |
336-ft/onnx/int8_dyn.onnx |
fine-tuned 336, dynamic-quantized INT8, CPU only; needs the ft336 thresholds; ~1.6× the FP32 speed on CPU at ¼ the size |
*-ft/onnx/ |
graphs + .data + the runtime config.json (re-swept thresholds), preprocessor_config.json, tag_zh.json — self-contained |
*-ft/safetensors/ |
the fine-tuned checkpoints (model.safetensors), each with the HF-style config.json (same thresholds), preprocessor_config.json, tagger_pipeline.py |
tagger_ort.py |
onnxruntime runner |
tagger_trt.py |
TensorRT runner (build an engine first) |
export_trt.py |
build engines: python export_trt.py build 672 [--fp32] [--workspace 2], fine-tuned graphs: python export_trt.py build --onnx 336-ft/onnx/fp16.onnx (engine next to the graph); export <size> re-exports from a checkpoint (--model-dir) |
config.json / preprocessor_config.json / tagger_pipeline.py |
zero-shot tag list (30,877), thresholds, processor (from the original repo) |
tag_zh.json |
Chinese names for the character (8,308) and copyright (2,460) tags — unused unless Tagger(zh=True) |
.gitattributes routes *.onnx, *.onnx.data, *.safetensors through LFS; the FP32
graphs need their .data sidecar next to them (relative filename, resolved by ORT and
by TensorRT).
Method. The oracle is the original FP32 checkpoint running at 1008 (PyTorch, no
autocast, batch 1). Tags are compared tag-by-tag above each model's own recommended
per-category thresholds (table below) on 1,000 real test images (WebP, long edge
500–2374) prepared independently of the fine-tuning data. A "flip" counts a tag crossing
its threshold in either direction; focus = character + copyright — the
identity-critical tags (~1,700 of them above threshold at 1008); the 15k+ general tags
are dense around their threshold by design and are the low-stakes category.
zs = the original weights with the pos/RoPE grids rebuilt for the target grid (no
training); ft = distilled from the oracle on 103,821 images (2 epochs).
| model | general | character | copyright | style | meta | rating |
|---|---|---|---|---|---|---|
| root (zs) | 0.17 | 0.27 | 0.24 | 0.15 | 0.17 | 0.41 |
| 336-ft | 0.32 | 0.54 | 0.60 | 0.37 | 0.22 | 0.05 |
| 672-ft | 0.32 | 0.60 | 0.59 | 0.44 | 0.18 | 0.06 |
The fine-tuned thresholds were re-swept on 3,000 held-out images; they are stricter and must be used with the fine-tuned graphs (the runners pick them up from the config next to the graph).
Focus-tag flips vs the oracle, per runtime:
| runtime | focus lost/gained | character | copyright | general flips |
|---|---|---|---|---|
| TRT FP16 | 1 / 1 | 0 / 1 | 1 / 0 | 50 / 54 |
| TRT FP32 (query-chunked) | 0 / 0 | 0 / 0 | 0 / 0 | 8 / 12 |
| ORT FP16 | 1 / 1 | 1 / 1 | 0 / 0 | 29 / 20 |
| ORT FP32 | 1 / 0 | 1 / 0 | 0 / 0 | 11 / 9 |
max |Δp| vs the checkpoint: 7.1e-3 (FP32) / 1.1e-2 (FP16), mean |Δp| ~1e-6 on both; every flipped tag sits right at its threshold (median baseline 0.170 vs the 0.17 general threshold, median movement 0.002, max ≤0.01). Safe default.
Focus-tag flips vs the oracle, fp32 (TensorRT engine; the onnxruntime and fp16 runs land within a few tags of these numbers, see Runtime consistency):
| model | focus lost/gained | character | copyright | general flips |
|---|---|---|---|---|
| 672 zs | 233 / 238 | 97 / 74 | 136 / 164 | 8,082 / 7,570 |
| 672-ft | 203 / 97 | 84 / 33 | 119 / 64 | 5,434 / 4,233 |
| 336 zs | 432 / 530 | 178 / 177 | 254 / 353 | 12,518 / 12,378 |
| 336-ft | 350 / 143 | 145 / 46 | 205 / 97 | 8,318 / 4,852 |
yae_miko 0.997 @1008 → 0.0002 @672; threshold_shift does not recover such losses —
they land far below any shift worth applying) — they are the reason the fine-tuned
checkpoints exist.The same model across the four runtimes (TRT/ORT × fp16/fp32) agrees closely: focus flips stay within ±3 tags and the dense general category within ±13. fp16 vs fp32 is identical on identity tags, so the differences above come from resolution and fine-tuning, not from the runtime or precision. fp16 is recommended on GPU; on GTX 16xx use fp32 (see Speed & memory).
192-image subset, int8_dyn.onnx vs the fp32 ORT path:
| metric | value |
|---|---|
| identity flips | 2 / 378 (0.5%) |
| general flips | 331 / 6,141 (5.4%, threshold-adjacent, low-stakes) |
| mean |Δp| | 7.2e-5 |
CPU-only (on GPU the quantized ops fall back to CPU kernels, ~3 img/s); ~1.6× the FP32 speed on CPU at ¼ the size. Must be used with the ft336 thresholds.
nvidia-cudnn-cu12 / nvidia-cublas-cu12 DLLs on top of
torch's bundled cuDNN makes ORT silently return wrong results (reproduced during this
testing). Let ORT resolve naturally (torch first, as both runners do), or run ORT in a
torch-free process.Rule of thumb: identity-critical tagging → 1008; bulk/backlog tagging → 336-ft (batch 2, fastest) or 672-ft when identity matters more and throughput less.
# Option A — onnxruntime (any CUDA GPU)
from tagger_ort import Tagger
tagger = Tagger(size=672) # zero-shot root graphs, FP16
tagger = Tagger(size=672, precision="fp32") # zero-shot FP32 graph
tagger = Tagger(size=336, ft=True) # fine-tuned: 336-ft/onnx/fp16.onnx (batch 2)
tagger = Tagger(size=448, ft=True) # 448-ft/onnx/fp16.onnx (batch 1, Hammy-only model)
tagger = Tagger(size=672, ft=True, precision="fp32")
r = tagger.tag("image.png") # {"character": {...}, "copyright": {...}}
r = tagger.tag(["a.png", "b.jpg", "c.png"]) # batched; 336 is fed two images at a time
# Option B — TensorRT (fastest): build an engine for your GPU first (under a minute):
# python export_trt.py build 672 (FP16, from tagger_672_fp16.onnx)
# python export_trt.py build --onnx 336-ft/onnx/fp16.onnx (-> 336-ft/onnx/fp16.trt, batch 2)
# --workspace caps device scratch in GiB (default 4) — use 2 on <=6 GB cards.
from tagger_trt import Tagger
tagger = Tagger(size=336, ft=True) # loads 336-ft/onnx/fp16.trt
tagger = Tagger(size=672, ft=True, precision="fp32")
model_dir=<folder> to point at any folder of your own: a folder holding
tagger_<size>_<tag>.onnx (+ config.json) works, and so does a -ft-style folder
(onnx/<tag>.onnx + onnx/config.json, checkpoint alongside in safetensors/).
ft=True is just shorthand for the repo's 336-ft/ / 672-ft/ folders.config.json next to the graph (root graphs:
repo root; fine-tuned: <size>-ft/onnx/config.json); pass threshold_shift=0.05 to
prune borderline tags.fp32.onnx graphs (dynamo-exported, with
query-chunked global attention so the engine fits 6 GB cards): TensorRT silently
miscompiles a legacy-TorchScript FP32 graph of this model, so never substitute one.tag_zh.json ships next to config.json (in the fine-tuned folders it sits in onnx/
with the graphs; the repo root copy is the fallback) and contains Chinese names for the
character and copyright tags only (no general tags), built for
Hammy; translation quality is not guaranteed. Both
runners ship with translation off by default — pass zh=True to enable it
(zh=True requires the file to exist):
tagger = Tagger(size=672, zh=True)
{"character": {"初音未来": 1.0}, "copyright": {"VOCALOID": 1.0}}
Pick one path, not all of them:
# A — onnxruntime only:
# pip install onnxruntime-gpu pillow numpy nvidia-cudnn-cu12 nvidia-cublas-cu12
# B — TensorRT engine, built on this machine from the shipped ONNX:
# pip install tensorrt nvidia-cuda-runtime nvidia-cuda-nvrtc --extra-index-url https://pypi.nvidia.cn
# (`tensorrt` itself has no win_amd64 wheel on PyPI — only NVIDIA's index carries the
# ~2 GB of binaries. `tensorrt` 11.3 unconditionally pulls `tensorrt_cu13`, a CUDA 13
# runtime — the driver must support CUDA 13 (>= 580). The wheels do NOT bundle
# cudart/nvrtc: nvinfer_11.dll imports nvcudart_hybrid64.dll and nvrtc64_130_0.dll,
# hence the two extra packages. Tested with TensorRT 11.3 on Windows; TRT 11
# supports sm_75 and up.)
# C — B, plus running the engine (tagger_trt.py imports the repo's tagger_pipeline.py):
# pip install torch --index-url https://download.pytorch.org/whl/cu128
# pip install transformers timm torchvision pillow
# Only `export_trt.py export <size>` needs a checkpoint (the original repo, or a
# `*-ft/safetensors` folder via --model-dir); `build` and both runners run from this
# repo alone.
INT8 needs onnxruntime CPU: pip install onnxruntime pillow numpy.
Converted from pixai-labs/pixai-tagger-v1.0
(© PixAI labs, Apache-2.0). The fine-tuned 336/448/672 checkpoints in *-ft/ were distilled
from that model's own outputs; this repository redistributes converted weights and runner
scripts under Apache-2.0; all model credit belongs to the original authors.
672-ft/safetensors/model.safetensors 1.9 GB448-Hammy/model.safetensors 1.9 GB448-ft/safetensors/model.safetensors 1.9 GB336-ft/safetensors/model.safetensors 1.9 GBtagger_1008_fp16.onnx 1.0 GB672-ft/onnx/fp16.onnx 997 MBtagger_672_fp16.onnx 997 MB448-Hammy/onnx/fp16.onnx 987 MB448-ft/onnx/fp16.onnx 987 MB336-ft/onnx/fp16.onnx 983 MB336-ft/onnx/int8_dyn.onnx 494 MBtagger_1008_fp32.onnx 397.7 kB448-Hammy/onnx/fp32.onnx 388.2 kB448-ft/onnx/fp32.onnx 388.2 kBtagger_672_fp32.onnx 386.4 kB672-ft/onnx/fp32.onnx 381.6 kB336-ft/onnx/fp32.onnx 373 kBpixai-labs/pixai-tagger-v1.0Have the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo HentaiSL/pixai-tagger-v1.0-onnx-1008-672-336-ft ./model-folder