Loading…
Loading…
GGUF quantizations of cross-encoder/ettin-reranker-150m-v1, a ModernBERT-based cross-encoder reranker (150M params, Apache-2.0).
Converted with CrispEmbed convert-bert-to-gguf.py.
| File | Quant | Size | Notes |
|---|---|---|---|
ettin-reranker-150m-v1-f32.gguf |
F32 | 572 MB | Full precision |
ettin-reranker-150m-v1-f16.gguf |
F16 | 288 MB | Half precision (identical scores to f32) |
ettin-reranker-150m-v1-q8_0.gguf |
Q8_0 | 155 MB | 8-bit (scores within 0.06) |
ettin-reranker-150m-v1-q4_k.gguf |
Q4_K | 102 MB | 4-bit (scores within 0.12) |
| Quant | France+ | France- | Pasta+ | Pasta- |
|---|---|---|---|---|
| Reference | +11.85 | -4.41 | +7.79 | -5.57 |
| F32 | +11.59 | -4.11 | +7.23 | -5.25 |
| F16 | +11.59 | -4.11 | +7.23 | -5.25 |
| Q8_0 | +11.57 | -4.10 | +7.19 | -5.30 |
| Q4_K | +11.50 | -4.13 | +7.17 | -5.35 |
Ranking is perfectly preserved across all quantizations.
Per-stage diff harness (f32 GGUF vs PyTorch): all 23 stages PASS, structural gate cos=1.000000, worst per-layer cos=0.999760.
# Rerank
crispembed -m ettin-reranker-150m-v1-q4_k.gguf --rerank-query "What is AI?" --rerank-doc "Artificial intelligence is..."
# C API
float score = crispembed_rerank(ctx, "query", "document");
# Batch rerank
int n = crispembed_rerank_batch(ctx, "query", docs, n_docs, scores);