KURE-Reranker-nano GGUF

GGUF of nlpai-lab/KURE-Reranker-nano (revision 8514b9e) for llama-server --reranking. License and model credit belong to the original authors (Apache-2.0).

F32 vs F16 on PublicHealthQA (77 queries × 50 documents): max logit difference 0.006, top-10 order changed in 1 query.

Run

llama-server -m KURE-Reranker-nano.F32.gguf --reranking --pooling rank -c 8192 -b 8192 -ub 8192
curl -s localhost:8080/v1/rerank -d '{"query":"대한민국의 수도는?","documents":["대한민국의 수도는 서울입니다.","바나나는 과일입니다."]}'

relevance_score is the raw logit. Apply a sigmoid for 0–1 scores.

Known differences from the original tokenizer

llama.cpp's WordPiece tokenizer differs from the original in two cases. Hangul, Latin, digits, punctuation and emoji tokenize identically.

• Hanja (CJK ideographs). llama.cpp always splits each ideograph into its own word (大韓民國 → 大 韓 民 國), while the original keeps word pieces (大 ##韓 ##民 ##國). Hanja-heavy text gets different scores and can change ranking.
• Non-NFC input. llama.cpp does not apply NFC for WordPiece. Decomposed Hangul (NFD, e.g. from macOS file names) breaks into jamo and scores lose contrast. Normalize to NFC before sending.

Conversion

The tokenizer is WordPiece, while llama.cpp's ModernBERT converter assumes GPT-2 BPE. convert_wordpiece_modernbert.py patches only the vocab step: it writes the WordPiece vocab with add_bos=true, add_eos=true, add_sep=false, and disables lowercasing and accent stripping. Input becomes <s> query </s> document </s>, the same layout as the original. Weights, the ModernBERT encoder, the mean-pooled classifier head and sliding-window attention go through the stock converter unchanged.

python convert_wordpiece_modernbert.py /path/to/llama.cpp KURE-Reranker-nano --outtype f32 --outfile KURE-Reranker-nano.F32.gguf

Converted with llama.cpp cd959ba (b12110) and tested on llama.cpp 0.9.0, Apple M5 Pro.

Verification

• Parity with the original (sentence-transformers): 400 pairs, Pearson 1.0000 and identical ranking for both F32 and F16. Tokenizer mismatches 0/210 on benchmark text.
• nDCG@10: reranker-simple-benchmark protocol (gold + BM25 top-50).

• Latency (Apple M5 Pro, Metal): 60 short documents 0.12 s (F32) / 0.11 s (F16), 300 documents 0.59 s / 0.55 s.