Mice-P-61m

This model is a MICE (Minimal Interaction Cross-Encoder) model based on jhu-clsp/ettin-encoder-68m (ModernBERT backbone).

MICE models separate query and document processing in the bottom layers, enabling independent document contextualization and faster re-ranking with cross-attention only in the top interaction layers.

Contents

• Model Description
• Usage
• Evaluations

Model Description

• Architecture: MICE v2 (Minimal Interaction Cross-Encoder)
• Base Model: jhu-clsp/ettin-encoder-68m
• Total Parameters: 61,328,385 (61.33M)
• Contextualization Layers: 10
• Interaction Layers: 4
• Training Strategy: MSE on mxbai-rerank-large-v2 (following the Ettin reranker Family recipe)
• Language: English

Usage

1. With sentence-transformers

from sentence_transformers import CrossEncoder

model = CrossEncoder("xpmir/Mice-P-61m", trust_remote_code=True)
scores = model.predict([
    ("What is the capital of France?", "Paris is the capital and most populous city of France.")
])
print(scores)

2. With Hugging Face transformers

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("xpmir/Mice-P-61m")
model = AutoModelForSequenceClassification.from_pretrained("xpmir/Mice-P-61m", trust_remote_code=True)
model.eval()

inputs = tokenizer(["What is the capital of France?"], ["Paris is the capital of France."], return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits
print(logits)

3. With xpmir

import torch
from xpmir.neural.sentence_transformers import STCrossEncoder
from xpmir.letor.records import PointwiseItems

scorer = STCrossEncoder.C(model_id="xpmir/Mice-P-61m").instance()
scorer.initialize()
scorer.eval()

queries = ["What is the capital of France?"]
documents = ["Paris is the capital and most populous city of France."]
input_records = PointwiseItems.from_texts(topics=queries, documents=documents)
with torch.no_grad():
    output = scorer(input_records)
print(output)

Evaluations

We provide evaluations of this cross-encoder re-ranking top 100 of naver/splade-v3-distilbert: