boboliu/Qwen3-Reranker-8B-W4A16-G128

🤗 Hugging Face sourcetext-classificationapache-2.08.2B params30 GBsafetensors✓ 3 checksumsupdated today

Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo boboliu/Qwen3-Reranker-8B-W4A16-G128 ./model-folder

Needs a seeder →

Qwen3-Reranker-8B-W4A16-G128

GPTQ Quantized Qwen/Qwen3-Reranker-8B with Ultrachat, THUIR/T2Ranking and m-a-p/COIG-CQIA for calibration set.

What's the benefit?

VRAM Usage: more than 24G -> 19624M, make it available on 3090/4090. (w/o FA2, according to Embedding model's result).

What's the cost?

I think <5% accuracy, further evaluation on the way...

The Embedding one shows ~0.7%.

How to use it?

pip install compressed-tensors optimum and auto-gptq / gptqmodel, then goto the official usage guide.