ONNX-optimized version of Qwen/Qwen3-Reranker-0.6B for use with qwen3-embed and fastembed.
| Variant | File | Size | Description |
|---|---|---|---|
| INT8 | onnx/model_quantized.onnx |
573 MB | Dynamic INT8 quantization (default) |
| Q4F16 | onnx/model_q4f16.onnx |
517 MB | INT4 weights + FP16 activations |
| YesNo INT8 | onnx/model_yesno_quantized.onnx |
572 MB | YES/NO-only logits (last token, 150x smaller output) |
| YesNo Q4F16 | onnx/model_yesno_q4f16.onnx |
517 MB | YES/NO-only logits + INT4 weights |
# INT8 (default)
from qwen3_embed import TextCrossEncoder
model = TextCrossEncoder("Qwen/Qwen3-Reranker-0.6B")
# Q4F16 (smaller, slightly less accurate)
model = TextCrossEncoder("Qwen/Qwen3-Reranker-0.6B-Q4F16")
onnxruntime.quantization.quantize_dynamic (QInt8)MatMulNBitsQuantizer (block_size=128, symmetric) + FP16 castonnx/model_quantized.onnx 601 MBonnx/model_yesno_quantized.onnx 600 MBonnx/model_q4f16.onnx 542 MBQwen/Qwen3-Reranker-0.6B