UNI-PIC/retail-qwen3-reranker-0.6b-tw

retail-qwen3-reranker-0.6b-tw is a reranker fine-tuned from Qwen/Qwen3-Reranker-0.6B for Traditional Chinese (Taiwan) retail product search. Given a shopper query and a product title, it scores how relevant the title is. It is designed as the second stage after a dense retriever such as UNI-PIC/retail-embeddinggemma-2-tw, which is itself a fine-tune of google/embeddinggemma-2.

Recommended use: rerank the top 50 candidates from the dense retriever and blend the reranker score with the retriever's own score (recipe below). On our held-out benchmark the blend improves nDCG@10 by about +0.03 over the retriever alone. The reranker on its own does not beat the fine-tuned retriever on the 39k-title benchmark catalog, so use it as a second stage, not a replacement.

Model Details

Key Features

• 🛒 Retail Search Adaptation: Trained on real shopper search queries whose candidate products were graded by an LLM judge, with judge-graded hard negatives from a retrieval stack.
• 🧩 Built for Blending: The reranker is most useful together with a dense retriever's score; the gain from the blend holds from a 39k-title catalog up to a 2.2M-title multi-source index.
• ⚡ Small: About 0.6B parameters. We measured roughly 700 to 770 (query, title) pairs per second on one H100 with plain Hugging Face bf16 inference (batch 64, no serving optimizations), so reranking 50 candidates costs about 65 to 70 ms of GPU time per query.
• 🇹🇼 Taiwan Query Style: Short brand and product-type keywords, mixed Chinese-English queries, and descriptive queries, with and without keyword tags.

Training Data

The training data is a proprietary Taiwan retail (zh-TW) dataset and is not publicly available. It consists of 20,413 listwise groups built from 8,560 real shopper search queries. Each group holds one positive product title and seven hard negatives. An LLM judge graded each (query, product title) pair as 0, 1, or 2; grade-2 products are positives and grade-0 products are negatives. Grade-1 products are never used as negatives, near-identical variants of a product are not paired as negatives, and titles that are negatives for many different queries are excluded. In 70% of the groups the query text carries keyword tags. Held-out evaluation queries were excluded from training.

Training Procedure

Evaluation

Evaluation Setup

The setup follows the retrieval benchmark of UNI-PIC/retail-embeddinggemma-2-tw. Queries are real shopper search queries that were never used in training (split by query group), restricted to queries with at least one relevant product in the judged pool: a development split (1,023 queries) and a held-out test split (368 queries) that was scored once. The first stage is the fine-tuned retriever (UNI-PIC/retail-embeddinggemma-2-tw, query + tags); the reranker scores its top 50 candidates and the blend is 0.5 * z(reranker) + 0.5 * z(retriever score), with z-scores computed within each query's candidate list. The blend weight is the untuned default. Scores here are not directly comparable with the table on the retriever's own card: the ideal ranking behind nDCG@10 is pooled over the systems compared in one run, so the embedding model alone scores slightly differently in each run.

Relevance is graded by an LLM judge (DeepSeek-V4-Flash) as 0, 1, or 2. Every product returned by any compared system is graded, so no system is penalized for retrieving a product that was not graded. Metrics are nDCG@10 against all graded products of each query and strict P@1 (the top-ranked product has grade 2); higher is better. Confidence intervals are 95% paired bootstrap intervals (5,000 resamples over queries).

🟢 improvement · 🔴 degradation · ⚪ no change (in the gain figures, 🟢 indicates that the 95% confidence interval excludes zero)

Recall of the first stage limits what any reranker can do: the top 50 candidates contain 94.5% (dev) and 94.9% (test) of the grade-2 products.

Held-out Test Split (368 queries, 39k-title catalog)

Embedding modelFine-tunedFine-tuned + embeddingEmbed.FTBlendΔ PP nDCG@1083.7%84.2%86.3%+2.7 P@182.1%81.5%85.9%+3.8

Development Split (1,023 queries, 39k-title catalog)

The stock Qwen3-Reranker-0.6B (with the same retail instruction and the same blend) is included as a control. It gains far less than the fine-tuned model, so the improvement comes from the fine-tuning and not from blending alone.

Embedding modelStock + embeddingFine-tunedFine-tuned + embeddingEmbed.StockFTBlendΔ PP nDCG@1085.4%85.0%84.6%87.5%+2.1 P@186.0%84.4%83.0%88.2%+2.2

Larger Multi-Source Index

To check that the gain does not depend on the small benchmark catalog, the 39k benchmark catalog was hidden inside a much larger index built from other proprietary Taiwan retail product catalogs (deduplicated by normalized title, 2.0M and 2.2M titles in total). Retrieval and reranking were repeated with the same top-50 reranking, and every newly retrieved product was judged.

Gain in nDCG@10 over the embedding model alone95% CI (paired bootstrap)Gain95% CI 0 5 PP Test · 39k catalog+2.7[1.2, 4.2] Dev · 39k catalog+2.1[1.4, 2.9] Test · 2.0M titles+3.0[2.2, 3.9] Test · 2.2M titles+3.5[2.7, 4.4] Dev · 2.0M titles+2.6[2.1, 3.1] Dev · 2.2M titles+2.6[2.1, 3.2]

Notes on reading this table:

• nDCG@10 uses an ideal ranking pooled over every judged product of the query, which now includes relevant products from the larger index. Because of that, nDCG@10 is only comparable within one row, not across index sizes. Within a row, the blend improves on the embedding model alone with a confidence interval that excludes zero.
• At 2M titles, more than 90% of the top-10 results come from outside the benchmark catalog. The judge sees titles only, so an equivalent product from another catalog can count as relevant.
• At this size the reranker alone has a higher P@1 than the 50/50 blend. The blend weight was not tuned; if P@1 matters most, tune the weight on your own data.
• The same table on the development split: gain of the blend over the embedding model alone in nDCG@10 is 39k +0.015, 2.0M +0.026, 2.2M +0.026.

Usage

The reranker uses the standard Qwen3-Reranker yes/no scoring (log-odds of yes versus no at the last position) with a retail instruction. rerank.py in this repository implements it together with the blend:

from rerank import RetailReranker

rr = RetailReranker("UNI-PIC/retail-qwen3-reranker-0.6b-tw")

query = "洗衣精"
titles = ["白蘭 洗衣精 補充包 1.6kg", "Hampton 漢汀堡 迪亞小圓桌 寶石藍", "一匙靈 抗菌洗衣精 2L"]
dense = [0.71, 0.32, 0.69]                       # similarity scores from the first-stage retriever

order, scores = rr.rank(query, titles, dense_scores=dense)    # blend (recommended)
order, scores = rr.rank(query, titles)                        # reranker score only
print([titles[i] for i in order])

In the evaluation, the retriever input was query + " " + tags when keyword tags were available, and the reranker saw the same text. Reranker inputs without tags were not separately evaluated.

Intended Use and Limitations

• Intended use: Second-stage reranking of 20 to 100 candidates for zh-TW retail product search, blended with a dense retriever's score.
• Candidate depth: Gains were measured at 50 candidates (recall of the first stage at 50 is about 95%); a depth of 20 gave a smaller gain in our tests.
• Judge-based evaluation: Training labels and benchmark labels come from the same family of LLM judge, and there are no human annotators or click data. Absolute scores should be read as comparisons between systems under this setup, not as production accuracy, and shared judge preferences may make the gain look somewhat larger than a human-judged gain.
• Modest gain: About +0.03 nDCG@10 over a strong fine-tuned retriever. It is not a general-purpose reranker and was not evaluated outside Taiwan retail titles.
• Single training catalog: Training used one proprietary catalog and its search logs. The larger-index experiment shows robustness at scale, not adaptation to other catalogs.
• Titles only: Documents were product titles; brand, price and description fields were not used.
• Language scope: Taiwan Traditional Chinese only.

License and Attribution

This model is released under the Apache License 2.0 (see LICENSE). It is a fine-tuned derivative of Qwen/Qwen3-Reranker-0.6B by the Qwen team, which is also licensed under Apache 2.0; all model weights were modified by fine-tuning.

Citation

@misc{retailqwen3reranker06btw,
  title     = {retail-qwen3-reranker-0.6b-tw},
  author    = {President Information Corp. (PIC) and Uni-President Information Philippines Corp. (UNI-PIPC)},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/UNI-PIC/retail-qwen3-reranker-0.6b-tw}
}

Contact

• Taiwan: pic.net.tw/contactUs
• Philippines: pic.net.ph/about/contactUs