🔎 KURE-Sparse-nano-inference-free

KURE-Sparse-nano-inference-free is the inference-free variant of KURE-Sparse-nano, a Korean-English learned sparse retrieval model. Documents are encoded by a compact (0.1B) sparse encoder, while queries need no model forward pass at all: a query is tokenized and each of its terms is looked up in a learned weight table. It averages 0.7360 nDCG@10 on MTEB(kor, v2) Retrieval, the highest among the inference-free sparse models we evaluated, at one third of KURE-Sparse-nano's retrieval FLOPs.

Key Characteristics

• Inference-free queries: query encoding is a tokenizer call and a table lookup, with no neural network involved, so it needs no GPU at serving time. On a single CPU thread, encode_query takes about 3 ms for one query and 0.3 ms per query in batches. Only documents go through the neural encoder, once, at indexing time.
• Learned Sparse Retrieval (LSR): every vector lives in a fixed vocabulary of 36,282 terms, so documents can be indexed in an inverted index and matched terms are inspectable.
• Compact: 0.1B-parameter document encoder.
• Long documents: up to 8,192 tokens.
• Cheap to search: on average 20 active dimensions per query and 958 per document, for 7.9 FLOPs per query-document pair (KURE-Sparse-nano: 23.0).

How It Works

• Documents are prefixed with [D] and encoded by a SPLADE-style encoder (log(1 + ReLU(logits)), max-pooled over tokens).
• Queries are tokenized, and every term present in the query gets that term's learned weight; a repeated term counts once, and special tokens are ignored. The table starts from each term's BM25 IDF over the training documents and is trained jointly with the document encoder.
• Terms: WordPiece continuations and their word-initial forms share one dimension on both sides (e.g. 계약 and ##계약), so a query term matches a document term however it was split.
• Score: the dot product, i.e. the sum over query terms of the term's query weight times its document weight.

The document encoder starts from the same backbone as KURE-Sparse-nano. It is contrastively pre-finetuned, then distilled from Qwen3-Reranker-8B on Korean and English retrieval data, with the query table trained throughout.

Usage

Sentence Transformers

pip install -U sentence-transformers

The model ships its own query and document modules (pooling.py), so load it with trust_remote_code=True. Use encode_query / encode_document: queries take the lookup route, and documents get the [D] marker and go through the encoder.

from sentence_transformers import SparseEncoder

model = SparseEncoder("nlpai-lab/KURE-Sparse-nano-inference-free", trust_remote_code=True)

query = "훈민정음은 언제 만들어졌나요?"
documents = [
    "세종대왕은 1443년에 훈민정음을 창제하고 1446년에 이를 반포하였다.",
    "김치는 배추나 무를 소금에 절인 뒤 고춧가루와 젓갈을 넣어 발효시킨 음식이다.",
    "한라산은 해발 1,947m로 남한에서 가장 높은 산이며 제주도 중앙에 자리한다.",
]

query_embeddings = model.encode_query(query)  # no model forward: tokenizer + lookup
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# torch.Size([36282]) torch.Size([3, 36282])

# Dot-product scores (higher is more relevant)
scores = model.similarity(query_embeddings, document_embeddings)
print(scores)
# tensor([[61.8513, 24.9416, 10.8340]])

Evaluation

KURE-Sparse-nano-inference-free is evaluated on the nine MTEB(kor, v2) Retrieval tasks. We report nDCG@10.

Performance-Sparsity

Average active dimensions per vector and FLOPs, averaged over the nine tasks.

Limitations

• No query expansion. A query only activates the terms it contains, so the model relies on the document side to cover paraphrases. On tasks with a large vocabulary gap between queries and documents it trails KURE-Sparse-nano most: LawIRKo (−0.126), MrTidy (−0.101) and MIRACL (−0.100).

Citation

@misc{kure-sparse,
  title  = {KURE-Sparse: a Korean-English bilingual learned sparse retriever},
  author = {Jang, Youngjoon and Son, Junyoung and Lee, Taemin and Hong, Seongtae and Lim, Heuiseok},
  year   = {2026},
  url    = {https://huggingface.co/nlpai-lab/KURE-Sparse-nano},
}

@misc{kure-v2,
  title  = {KURE-v2: a Korean-English bilingual late-interaction retriever},
  author = {Jang, Youngjoon and Son, Junyoung and Lee, Taemin and Hong, Seongtae and Lim, Heuiseok},
  year   = {2026},
  url    = {https://huggingface.co/nlpai-lab/KURE-v2},
}

@inproceedings{jang2025kure,
  title={KURE: Embedding Model for Korean-Specific Retrieval},
  author={Jang, Youngjoon and Son, Junyoung and Lee, Taemin and Hong, Seongtae and Park, JeongBae and Lim, Heuiseok},
  booktitle={Annual Conference on Human and Language Technology},
  pages={129--134},
  year={2025},
  organization={Human and Language Technology}
}