🇰🇷 한국어 카드: MODEL_CARD.ko.md

EG2-Text-Hybrid-129M

A mixed-precision text tower of EmbeddingGemma 2 that is 15% smaller than our 4-bit release (eg2-text-q4) while staying within 25% of that release's quality gap to uniform 3-bit on every held-out slice we measured.

"129M" refers to the size of the weights file in megabytes, not to a parameter count.

What is in the file

The idea is an allocation choice rather than a new quantizer. EmbeddingGemma 2's 262,144-row vocabulary table is about half of the text tower's weights. Spending uniform 3 bits on it wastes budget on rows that rarely matter. This checkpoint keeps the transformer at 4 bits, drops the vocabulary to 2 bits, and restores 4 bits on the 30% of rows that score highest on token frequency × 2-bit reconstruction error.

Token frequencies were counted on the FLORES-200 dev split (all 204 languages) plus an internal Korean corpus. None of the evaluation text was used. The selected rows were frozen before any held-out evaluation was run.

Mixed precision as a general idea is well established. What this release contributes is a specific allocation for this model, measured against uniform baselines at matched byte budgets.

Evaluation

Every number is asymmetric: quantized query embeddings are searched against a BF16 corpus index. That is the setting this checkpoint is for — replacing the query encoder of a system whose index is already built, without re-indexing.

The held-out set was built and hashed before the configuration was chosen, and used exactly once.

Slices

• Multilingual: FLORES-200 devtest sentences that were not used during development, as queries in 16 languages, retrieving their English source. 9,792 queries, 1,012-document pool.
• Code: HumanEval docstring ↔ function, in both directions. 164 each.
• Rare identifiers: templated queries built around tokenizer pieces stratified across the vocabulary. Same-template documents that differ only in the identifier act as hard negatives. 1,000 queries.

Protocol

• All arms ran in the same job, on the same GPU, against the same pool.
• Each quantized arm was built 6 times: different group offsets for RTN, different calibration orders for GPTQ.
• Tables report the median.
• A difference counts as real only if it exceeds twice the larger build-to-build spread of the two arms.

The table cells are margin / neighbor retention @10.

What the comparison supports

• Against uniform 3-bit (10 MB smaller), this release is better on all 8 cells, and each difference exceeds 2× the build spread.
• Against the 4-bit RTN recipe (23 MB larger), no cell shows a regression beyond 2× the build spread. Measured as a fraction of the 3-bit → 4-bit gap, the loss is at most 0.233. That worst case is rare-identifier margin. Multilingual, both code directions and rare-identifier retention stay within 0.16.
• Row selection matters. Restoring a random 30% of rows at the same size recovers much less, for example multilingual retention of 0.750 against 0.850.

On the internal Korean retrieval set used for eg2-text-q4 (n = 100), the same job measured 0.1731 / 0.946 for this release and 0.1723 / 0.940 for 4-bit RTN. That set was used during development, so it is not held-out evidence.

Definitions

• Margin is the mean gap between a query's positive similarity and its hardest negative.
• Neighbor retention @10 is the overlap between the arm's top-10 BF16-corpus neighbors and those of the BF16 query, divided by 10.

The evaluation data and harness are internal and not released.

Cross-modal compatibility

This checkpoint is a text query encoder. The question that matters for it is whether its queries still search a frozen BF16 EmbeddingGemma 2 index built from images, video and audio by the upstream model. We measured exactly that: the corpus side was BF16 throughout, and only the text query encoder changed.

The verdict rule, the floor definition and the per-lane decision were registered before the run. The floor is the spread across 6 rebuilds of this checkpoint's recipe (different GPTQ calibration seeds), which is the variance a user actually faces. A difference counts only if it exceeds twice that floor and its paired bootstrap CI excludes zero.

What this supports. Compatible with frozen BF16 EmbeddingGemma 2 multimodal indexes across image, video and audio retrieval. Neighbor@10 preservation was not distinguishable from the Q4 query encoder in any evaluated lane, while small measurable margin regressions remained on image retrieval, particularly multilingual image retrieval.

What it does not support. This is not "equal to Q4 on every modality". On both image lanes the margin loss against Q4 is real, and on the 15-language lane it is 4.4× the floor. Multilingual image retrieval is the weakest case we found, which is consistent with the row-selection corpus being FLORES rather than image captions.

The audio lane carries 36 queries (18 classes × 2 languages). Read its "no regression" as insufficient power to tell, not as evidence of equality.

Corpora: an internal Korean stock-image set, Wikipedia image captions in 15 languages, an internal Korean background-video set, and FSD50K (CC0). The frozen index for each lane is pinned by a manifest with the corpus hash, encoder revision and preprocessing, so a later checkpoint can be measured against the same index.

Usage

⚠️ AutoModel.from_pretrained alone will not load this checkpoint. eg2_quant.py in this repository dequantizes the packed codes and builds a standard EmbeddingGemma 2 text model.

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("ThakiCloud/eg2-text-hybrid-129m")
sys.path.insert(0, path)
from eg2_quant import load_quantized_text_model, QUERY_PREFIX, DOC_PREFIX

model, processor = load_quantized_text_model(path)

Use EmbeddingGemma 2's prefixes ("task: search result | query: " for queries and "title: none | text: " for documents), mean pooling, then L2 normalization. If you truncate for Matryoshka use, re-normalize afterwards.

The loader checks that every embedding row is claimed exactly once, and fails loudly otherwise. Decoding is exact: the reloaded weights match the in-memory quantized model bit for bit.

What this release does not claim

• Not a 3-bit model. It is mixed 2/4-bit at about 3.8 effective bits per parameter. Do not compare it to "Q3" checkpoints by name.
• Not serving-validated. No inference endpoint has been run on it. There are no latency, memory or device claims.
• The multilingual result may be optimistic. Row selection counted tokens on FLORES-200 dev, which comes from the same distribution as the multilingual test sentences, although the sentences differ. Languages or domains far from FLORES may recover less.
• Rare identifiers are the weakest case. Tokens absent from the frequency corpus are restored only through the reconstruction-error term.
• Small code set. n = 164.
• Calibration language. GPTQ calibration used Korean text only.
• Not independently reproducible. The data and harness are internal.
• Text only. The vision and audio towers are not in the file.

Related papers

Two measurement papers sit behind this line's bit allocation. Both quantize EmbeddingGemma-300M, the previous generation; EmbeddingGemma 2 is in neither grid, so read them as the prior that shaped this release rather than as evidence about this checkpoint.

• Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families, arXiv:2609.16391 — the received advice for shrinking a retrieval embedder fails to transfer as stated. The embedding table never emerges as the dominant isolated protection priority in any family, and at INT4/g16 the spread between modules is too small to allocate against.
• Quantize by Drift: Label-Free Mixed-Precision Post-Training Quantization for Text Embedders, arXiv:2610.09227 — quantization-induced representation drift, the same quantity this card's neighbor-retention numbers measure, used as a label-free sensitivity signal. It orders mixed-precision plans at a macro Spearman of 0.911, but at the main budget it does not beat same-budget uniform precision on sealed held-out embedders.

A third paper concerns calibration rather than allocation: When PTQ Calibration Helps or Hurts: Dose-Driven Collapse and Validation-Gated Abstention in Quantized Text Embedders, arXiv:2610.27397. It fits quantizer parameters on a calibration set at 4-bit for e5-base-v2, BGE-M3 and EmbeddingGemma-300M, and finds the outcome is set by the dose of the update: on EmbeddingGemma-300M the same learning-rate grid spans +1.47 to -71.53 points of in-domain NDCG@10. Its validation-gated accept rule never chose a checkpoint below the uncalibrated baseline (0 harmful selections in 30 decisions), but it is a safety trade, not a free gain: mean oracle regret 1.52 pp. This checkpoint's GPTQ pass reads a calibration set but fits no quantizer parameters by gradient, and EmbeddingGemma 2 is not in that grid.

The 2-bit vocabulary with 4-bit rows restored here is an allocation measured on this model, not an instance of a general rule, and the frequency × reconstruction-error score it uses is the label-dependent alternative to the drift signal in the second paper.

License and attribution

Apache License 2.0, inherited from the base model. A copy is included as LICENSE.

Modifications to the base model:

• Transformer linear layers were quantized with 4-bit GPTQ.
• The token embedding table was quantized to 2-bit, with 30% of rows kept at 4-bit RTN.
• The vision and audio towers were removed.
• No weights were trained or fine-tuned.

Base model © Google DeepMind. This is an independently quantized derivative and is not endorsed by Google.