🇰🇷 한국어 카드: MODEL_CARD.ko.md

EG2-Text-Hybrid-119M

The size-oriented member of our EmbeddingGemma 2 text-tower family. At the byte budget of a uniform 3-bit checkpoint it is measurably better than uniform 3-bit, and it gives up a measurable part of 4-bit quality to get there.

"119M" is the size of the weights file in megabytes, not a parameter count.

Read this first. If 119 MB is not a hard constraint, use eg2-text-hybrid-129m (10 MB larger) or eg2-text-q4 (33 MB larger) instead. On the multilingual slice below, neighbor retention @10 is 0.707 here against 0.848 for the 4-bit recipe. This checkpoint exists for the case where the file has to fit under 120 MB.

What is in the file

The allocation is the whole idea. EmbeddingGemma 2's vocabulary table holds about half of the text tower's weights, so a uniform 3-bit checkpoint spends the same budget per parameter on the transformer and on the vocabulary. This checkpoint spends 4 bits on the transformer and 2 bits on the vocabulary, which lands at the same file size and, on this held-out set, scores better.

Mixed precision as a general technique is well established. What is specific here is the allocation for this model, measured against uniform baselines at a matched byte budget.

This is the simplest member of the family. The 129 MB release adds a selective row rescue on top: it restores 4 bits on the 30% of vocabulary rows that score highest on token frequency times 2-bit reconstruction error. If you can afford those 10 MB, that rescue is where most of the quality comes back.

Evaluation

Every number is asymmetric: quantized query embeddings are searched against a BF16 corpus index. That is what this checkpoint is for — replacing the query encoder of a system whose index already exists, without re-indexing.

The held-out set was built and hashed before the configurations were chosen, and used exactly once.

Slices

• Multilingual: FLORES-200 devtest sentences held out from development, as queries in 16 languages retrieving their English source. 3,520 queries.
• Code: HumanEval docstring ↔ function, both directions. 489 each.
• Rare identifiers: templated queries built around tokenizer pieces stratified across the vocabulary, with same-template hard negatives. 511 queries.

Protocol

• All arms ran in one job, on one GPU, against the same pool.
• Each arm was built 6 times — different group offsets for RTN, different calibration orders for GPTQ.
• Tables report the median across builds.
• A difference counts only if it exceeds twice the larger build-to-build spread of the two arms.

Cells are margin / neighbor retention @10.

What the comparison supports

• Against uniform 3-bit at the same budget, this release is better on all 8 cells, and 6 of them exceed 2× the build spread. No cell regresses (0 negative signs).
• Against the 4-bit recipe (152,549,424 bytes of payload), the loss is real, not borderline. Read as a fraction of the 3-bit → 4-bit gap, it ranges from 0.31 on the code slices to 0.73 on multilingual and rare identifiers. Most of the 4-bit advantage on multilingual retrieval is gone.
• The GPTQ transformer helps a little. The same allocation with an RTN transformer is slightly behind on 7 of 8 cells, but no single cell clears 2× the build spread, so treat that as directional.

On the internal Korean retrieval set used during development (n = 100), the same job measured 0.1391 / 0.878 for this release and 0.1723 / 0.940 for the 4-bit recipe. That set is not held-out evidence.

Definitions

• Margin is the mean gap between a query's positive similarity and its hardest negative.
• Neighbor retention @10 is the overlap between this arm's top-10 BF16-corpus neighbors and the BF16 query's own top-10, divided by 10.

The evaluation data and harness are internal and not released.

Usage

⚠️ AutoModel.from_pretrained alone will not load this checkpoint. eg2_quant.py in this repository unpacks the codes and builds a standard EmbeddingGemma 2 text model.

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("ThakiCloud/eg2-text-hybrid-119m")
sys.path.insert(0, path)
from eg2_quant import load_quantized_text_model, QUERY_PREFIX, DOC_PREFIX

model, processor = load_quantized_text_model(path)

Use EmbeddingGemma 2's prefixes ("task: search result | query: " for queries, "title: none | text: " for documents), mean pooling, then L2 normalization. If you truncate for Matryoshka use, re-normalize afterwards.

Decoding is exact: reloaded weights match the in-memory quantized model bit for bit.

What this release does not claim

• Not a 3-bit model. It is mixed 2/4-bit at about 3.51 effective bits per parameter. Do not line it up against "Q3" checkpoints by name.
• Not equal to 4-bit. The gap is measurable on multilingual and rare-identifier retrieval. This is a size trade, and the card states the price.
• Not comparable cell-by-cell with the 129 MB card. That release was measured on a second, separate held-out set, so the two tables are not the same measurement. Both were measured against the same 4-bit arm inside their own job, which is the only sound way to read them together.
• Not serving-validated. No inference endpoint has been run on it. No latency, memory or device claims.
• Cross-modal compatibility is untested for this checkpoint. The image, video and audio compatibility lanes we ran were on the 129 MB release, and image retrieval showed a small measurable margin regression there. Nothing was measured here.
• The multilingual slice may be optimistic. GPTQ calibration used Korean text only, and the slice is FLORES, which is a narrow domain.
• Small code set. n = 489.
• Not independently reproducible. The data and harness are internal.
• Text only. The vision and audio towers are not in the file.

Related papers

Two measurement papers sit behind this line's bit allocation. Both quantize EmbeddingGemma-300M, the previous generation; EmbeddingGemma 2 is in neither grid, so read them as the prior that shaped this release rather than as evidence about this checkpoint.

• Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families, arXiv:2609.16391 — the received advice for shrinking a retrieval embedder fails to transfer as stated. The embedding table never emerges as the dominant isolated protection priority in any family, and at INT3 the module ordering becomes family-dependent while joint damage stops being the sum of its parts. That is the regime this checkpoint sits in.
• Quantize by Drift: Label-Free Mixed-Precision Post-Training Quantization for Text Embedders, arXiv:2610.09227 — a label-free alternative to this card's row-selection score: quantize one module, re-encode the corpus, and measure how far the output embeddings moved. It orders mixed-precision plans at a macro Spearman of 0.911, yet at the main budget it lands below same-budget uniform on three sealed embedders, which is why beating uniform 3-bit at the same bytes is the claim this card makes and the only one it makes.

A third paper concerns calibration rather than allocation: When PTQ Calibration Helps or Hurts: Dose-Driven Collapse and Validation-Gated Abstention in Quantized Text Embedders, arXiv:2610.27397. It fits quantizer parameters on a calibration set at 4-bit for e5-base-v2, BGE-M3 and EmbeddingGemma-300M, and finds the outcome is set by the dose of the update: on EmbeddingGemma-300M the same learning-rate grid spans +1.47 to -71.53 points of in-domain NDCG@10. Its validation-gated accept rule never chose a checkpoint below the uncalibrated baseline (0 harmful selections in 30 decisions), but it is a safety trade, not a free gain: mean oracle regret 1.52 pp. This checkpoint's GPTQ pass reads a calibration set but fits no quantizer parameters by gradient, and EmbeddingGemma 2 is not in that grid.

License and attribution

Apache License 2.0, inherited from the base model. A copy is included as LICENSE.

Modifications to the base model:

• Transformer linear layers were quantized with 4-bit GPTQ.
• The token embedding table was quantized to 2-bit RTN.
• The vision and audio towers were removed.
• No weights were trained or fine-tuned.

Base model © Google DeepMind. This is an independently quantized derivative and is not endorsed by Google.