UNI-PIC/retail-qwen3-reranker-0.6b-tw-fast

retail-qwen3-reranker-0.6b-tw-fast is the fast sibling of UNI-PIC/retail-qwen3-reranker-0.6b-tw: a reranker fine-tuned from Qwen/Qwen3-Reranker-0.6B for Traditional Chinese (Taiwan) retail product search that does the same job about 1.8x faster. We did not shrink the model, we shrank the question: each (query, title) pair is wrapped in a prompt of about 41 tokens instead of about 111, and the model was retrained to read that short format. It is the second stage after a dense retriever such as UNI-PIC/retail-embeddinggemma-2-tw.

Recommended use: rerank the top 50 candidates from the dense retriever and blend the reranker score with the retriever's own score (recipe below). In our benchmarks this blend keeps most of the standard reranker's gain (87% to 96%, depending on the judge and split) at 1.8x the throughput. The fast reranker on its own is weaker than the standard one, so use it as a blended second stage, not a replacement for the retriever.

Model Details

Key Features

• ⚡ About 1.8x faster: 1,691 versus 952 (query, title) pairs per second end to end on one H100 (median of 5 passes, details below), which is about 34 versus 19 searches per second per GPU when each search reranks 50 candidates (44% less reranking time per search). Measured on one NVIDIA H100 with plain Hugging Face bf16 inference, batch 64, no serving optimizations.
• 🧠 Same quality where it matters: In the blend, the fast reranker is statistically tied with the standard reranker on every split and index size we tested, and keeps 87% to 96% of its gain over the embedding model alone, as measured by two independent LLM judges.
• 🧩 Drop-in recipe: Same model size, same scoring rule (log-odds of yes versus no at the last position), same blend. Only the prompt template changes, so it must be used with the short template in rerank.py.
• 🛒 Retail Search Adaptation: Trained on real shopper search queries whose candidate products were graded by an LLM judge, with judge-graded hard negatives from a retrieval stack.
• 🇹🇼 Taiwan Query Style: Short brand and product-type keywords, mixed Chinese-English queries, and descriptive queries, with and without keyword tags.

How System 1 Works

A reranker reads every candidate in full: for each search it scores 50 (query, title) pairs, one forward pass each. The model is small, so the time is dominated by how many tokens it has to read per pair. The standard Qwen3-Reranker prompt spends most of those tokens on text that never changes: a system prompt, a task instruction and an empty think block. System 1 removes that fixed text and trains the model to work from the bare query and title.

WHERE THE FAST RERANKER SITSShopper querye.g. 水壺 500mlEmbedding modelfinds the top 50candidate titlesFast rerankerreads all 50query-title pairsBlend50% reranker +50% embedding scoreRanked resultsbest productat the topThe reranker runs 50 times for every search, so its cost per pair decides the latency.System 1 cuts that cost by shrinking what the model has to read per pair (next diagram). FROM QWEN3-RERANKER TO SYSTEM 1: SHRINK WHAT THE MODEL READS, NOT THE MODELStandard promptabout 111 tokens per query-title pairFixed boilerplate, the same for every pairsystem prompt · task instruction · empty think blockquery + titleSystem 1 promptabout 41 tokens per pair, about 63% fewerquery + titlesame model, same 0.6B weights,same yes/no scoring ruleHOW WE GOT THERE1New prompt templateA ~20-line ms-swift plug-in writes theshort format; a check script confirms itmatches the benchmark prompt token fortoken.2Retrain, nothing else changedSame fine-tuned recipe and the same 20,413judged groups, 1 epoch, 626 steps, about 15min on one H100. The model re-learns theshort format.3Check two gatesAt least 1.6x faster end to end and atleast 80% of the standard reranker's gain.Result: 1.78x faster, 87% to 96% of thegain.Unchanged: model size and architecture, training data, loss, and how the score is read (last-token yes/no logit).

The short prompt is exactly:

<|im_start|>user
<Query>: {query}
<Document>: {title}<|im_end|>
<|im_start|>assistant

The model's answer is read from the last position as logit(yes) - logit(no), as in Qwen3-Reranker. Because the weights were trained on this template, other templates give poor scores.

Training Data

The training data is a proprietary Taiwan retail (zh-TW) dataset and is not publicly available. It consists of 20,413 listwise groups built from 8,560 real shopper search queries (all real queries, no synthetic data). Each group holds one positive product title and seven hard negatives. An LLM judge graded each (query, product title) pair as 0, 1, or 2; grade-2 products are positives and grade-0 products are negatives. Grade-1 products are never used as negatives, near-identical variants of a product are not paired as negatives, and titles that are negatives for many different queries are excluded. In 70% of the groups the query text carries keyword tags. Held-out evaluation queries were excluded from training. The data and recipe are identical to the standard reranker's; only the prompt template differs.

Training Procedure

Evaluation

Evaluation Setup

The setup follows the retrieval benchmark of UNI-PIC/retail-embeddinggemma-2-tw and of the standard reranker's card. Queries are real shopper search queries that were never used in training (split by query group), restricted to queries with at least one relevant product in the judged pool: a development split (1,023 queries) and a held-out test split (368 queries) that was scored once. The first stage is the fine-tuned retriever (UNI-PIC/retail-embeddinggemma-2-tw, query + tags); the reranker scores its top 50 candidates and the blend is 0.5 * z(reranker) + 0.5 * z(retriever score), with z-scores computed within each query's candidate list. The blend weight is the untuned default. In every table the standard and fast rerankers were scored in the same run on the same candidates. nDCG@10 uses an ideal ranking pooled over the systems compared in one run, so compare numbers only within a table.

Relevance is graded by an LLM judge (DeepSeek-V4-Flash) as 0, 1, or 2. Every product returned by any compared system is graded, so no system is penalized for retrieving a product that was not graded. Metrics are nDCG@10 against all graded products of each query and strict P@1 (the top-ranked product has grade 2); higher is better. Confidence intervals are 95% paired bootstrap intervals (5,000 resamples over queries). The held-out test split was additionally graded by a second, independent judge (Qwen3.5-122B, run locally) to check that the result does not depend on one judge.

🟢 improvement · 🔴 degradation · ⚪ no change (in the gain figures, 🟢 indicates that the 95% confidence interval excludes zero)

Held-out Test Split (368 queries, 39k-title catalog), first judge

Embedding modelStandard + embeddingFast + embeddingEmbed.StdFastΔ PP nDCG@1083.9%86.5%86.3%+2.4 P@182.1%86.1%85.3%+3.3

Held-out Test Split, second judge (Qwen3.5-122B)

The same 368 test queries and the same systems, graded by an independent judge. Both rerankers' gains hold, and the fast reranker keeps 87% of the standard reranker's blend gain.

Development Split (1,023 queries, 39k-title catalog)

Embedding modelStandard + embeddingFast + embeddingEmbed.StdFastΔ PP nDCG@1072.3%74.0%73.9%+1.6 P@186.5%88.4%88.2%+1.7

Larger Multi-Source Index (development split)

To check that the gain does not depend on the small benchmark catalog, the 39k benchmark catalog was hidden inside a much larger index built from other proprietary Taiwan retail product catalogs (deduplicated by normalized title, 2.0M and 2.2M titles in total). Retrieval and reranking were repeated with the same top-50 reranking, and every newly retrieved product was judged. This check was run on the development split only for the fast reranker.

Gain in nDCG@10 vs embedding model95% CI (paired bootstrap)Gain95% CI 0 5 PP Test · 39k, judge 1+2.4[0.9, 4.0] Test · 39k, judge 2+1.9[0.7, 3.3] Dev · 39k catalog+1.6[1.0, 2.3] Dev · 2.0M titles+2.5[2.0, 3.1] Dev · 2.2M titles+2.7[2.2, 3.2]

Notes on reading this table:

• nDCG@10 is only comparable within one row, not across index sizes, because the pooled ideal ranking changes with the index.
• The judge sees titles only, so an equivalent product from another catalog can count as relevant.
• At this size the reranker alone can have a higher P@1 than the 50/50 blend; the blend weight was not tuned, so tune it on your own data if P@1 matters most.

Speed

How this was measured: 20,000 real (query, title) pairs, one NVIDIA H100, plain Hugging Face bf16 inference with no serving optimizations, batch size 64 with length-sorted batches, both models loaded in the same process and timed alternately for 5 passes (median reported; the end-to-end range was 948-957 versus 1,670-1,715). The 50-candidate figure is 50 divided by the throughput; real per-query batches of 50 will differ with your serving stack. Single passes inside the quality benchmarks gave 1.6x to 1.7x. Treat the ratio, not the absolute numbers, as the transferable result.

Usage

rerank.py in this repository implements the short template, the yes/no scoring and the blend:

from rerank import RetailReranker

rr = RetailReranker("UNI-PIC/retail-qwen3-reranker-0.6b-tw-fast")

query = "洗衣精"
titles = ["白蘭 洗衣精 補充包 1.6kg", "Hampton 漢汀堡 迪亞小圓桌 寶石藍", "一匙靈 抗菌洗衣精 2L"]
dense = [0.71, 0.32, 0.69]                       # similarity scores from the first-stage retriever

order, scores = rr.rank(query, titles, dense_scores=dense)    # blend (recommended)
order, scores = rr.rank(query, titles)                        # reranker score only (weaker than the blend)
print([titles[i] for i in order])

In the evaluation, the retriever input was query + " " + tags when keyword tags were available, and the reranker saw the same text. Reranker inputs without tags were not separately evaluated. Do not use the standard Qwen3-Reranker prompt with this model.

Intended Use and Limitations

• Intended use: Second-stage reranking of 20 to 100 candidates for zh-TW retail product search, blended with a dense retriever's score, where reranking latency or GPU cost matters.
• Use the blend: On the held-out test split the fast reranker alone is slightly weaker than the standard reranker alone and does not beat the embedding model alone. Its value is as a blended second stage.
• Speed is workload dependent: The ratio comes from one GPU and one batching setup. Gains are smaller when text preparation, network or retrieval dominate your latency.
• Candidate depth: Gains were measured at 50 candidates; shallower candidate lists give smaller gains.
• Judge-based evaluation: Training labels and benchmark labels come from LLM judges, and there are no human annotators or click data. Absolute scores should be read as comparisons between systems under this setup, not as production accuracy.
• Modest, tied quality: The fast reranker is statistically tied with the standard one in the blend, not better. Its retained share of the gain is 87% to 96% depending on judge and split.
• Limited larger-index test: The 2.0M and 2.2M title checks were run on the development split only for this model.
• Single training catalog: Training used one proprietary catalog and its search logs. It was not evaluated outside Taiwan retail titles.
• Titles only, Traditional Chinese only: Documents were product titles; brand, price and description fields were not used.

License and Attribution

This model is released under the Apache License 2.0 (see LICENSE). It is a fine-tuned derivative of Qwen/Qwen3-Reranker-0.6B by the Qwen team, which is also licensed under Apache 2.0; all model weights were modified by fine-tuning.

Citation

@misc{retailqwen3reranker06btwfast,
  title     = {retail-qwen3-reranker-0.6b-tw-fast},
  author    = {President Information Corp. (PIC) and Uni-President Information Philippines Corp. (UNI-PIPC)},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/UNI-PIC/retail-qwen3-reranker-0.6b-tw-fast}
}

Contact

• Taiwan: pic.net.tw/contactUs
• Philippines: pic.net.ph/about/contactUs