🇰🇷 한국어 카드: MODEL_CARD.ko.md

EG2 Hybrid — BF16 Text + W4 Media

EmbeddingGemma 2 with BF16 text weights and 4-bit vision/audio weights. The bundle is 57% of the upstream BF16 repository size on disk. Image, audio, and video retrieval met pre-registered non-inferiority criteria on held-out evaluation sets; the text tower remains unchanged.

What this saves, and what it does not

This is a disk and download reduction. The loader in this repository dequantizes the 4-bit weights back to BF16 when the model is loaded, so memory after loading is the same as the BF16 original, and an inference throughput improvement has not been reliably established:

Peak GPU memory while loading is 3,316,306,432 bytes, against 1,501,874,176 bytes for upstream BF16 (the loader builds the model once to dequantize and once as a SentenceTransformer, then copies the weights across), and loading takes 26.2 s against 1.9 s for upstream BF16 (same job, same GPU). If you need a smaller memory footprint at run time, this release does not provide it.

Evaluation

Every number below compares this bundle against upstream BF16 on the same queries, in the same job, on the same GPU. The primary metric is Recall@1 when the bundle is used on its own (bundle queries against a bundle-built index). The decision rule was fixed before any of these sets was scored: a modality passes as non-inferior if the lower end of the 95% confidence interval of ΔR@1 stays above −δ, with δ = 10% of that modality's BF16 R@1.

Intervals are 95% cluster bootstrap (image: near-duplicate component, audio: sound class, video: source broadcast). We do not read the positive audio estimate as an improvement: 101 queries is small and the interval includes zero.

The text tower is not modified: across all three evaluation jobs, text-query embeddings from the bundle matched upstream BF16 with minimum cosine 0.9999998. Because image, audio and video embeddings pass through the text backbone after their own tower, a quantized media tower is the only source of difference.

About the evaluation data. The image and video sets come from Korean public datasets distributed by AI Hub (aihub.or.kr) under terms that do not allow redistribution, and the audio set is built from FSD50K. None of these sets were used to build or tune this bundle, and they are not released with it. The numbers are therefore internally measured with a pre-registered protocol and are not third-party reproducible.

Usage

Do not load this repository with SentenceTransformer("…") or AutoModel.from_pretrained("…") directly — it will not load correctly: the media weights are packed integer codes plus group scales. eg2_quant.py in this repository dequantizes them and returns an ordinary SentenceTransformer.

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("ThakiCloud/eg2-hybrid-w4-media")
sys.path.insert(0, path)
from eg2_quant import load_sentence_transformer

model = load_sentence_transformer(path)          # BF16 on GPU if available

Encode each modality the same way as upstream EmbeddingGemma 2. Normalize after encoding (and again after any Matryoshka truncation).

import numpy as np
from PIL import Image

def unit(x):
    return x / np.linalg.norm(x, axis=-1, keepdims=True)

q = unit(model.encode(["a red circle on a light background"], prompt_name="SearchQuery"))
img = unit(model.encode([{"image": Image.open("photo.jpg")}]))
aud = unit(model.encode([{"audio": waveform_16k_mono}]))           # float32 numpy array, 16 kHz mono
vid = unit(model.encode([{"video": list_of_pil_frames}]))          # e.g. 1 frame per second
scores = q @ np.concatenate([img, aud, vid]).T

Examples (synthetic inputs, measured)

To show what changes and what does not without shipping any third-party content, we encoded deterministic synthetic inputs with both this bundle and upstream BF16 and compared the embeddings directly.

Text is unchanged by construction. Media embeddings move slightly (cosine about 0.996 to 0.999), which is what the retrieval table above measures at scale. Repeated encodes with the bundle were deterministic (minimum cosine 1.0000).

What this release does not claim

• No run-time memory saving, and no established inference speed-up — see above. Loading is slower and needs more peak memory than upstream.
• No modality-selective loading. Upstream EmbeddingGemma 2 can load only the encoders you need; this loader always builds the full model, including both media towers.
• Not the only or best configuration. Other text/media precision combinations were built; this card makes no claim that this one is optimal among them.
• Not serving-validated. No inference endpoint has been started on this checkpoint.
• Not independently reproducible. Evaluation data and harness are internal.
• Languages other than Korean and English, and media domains other than those above, are unmeasured.

Limitations

The 4-bit media towers keep about 86% (image) and 83% (video) of the BF16 top-10 neighborhood when the index is built with the bundle, so applications that depend on exact neighbor identity — deduplication, near-duplicate or contamination checks — should validate on their own data. Mixing indexes is a separate case: an index built with upstream BF16 and queried with this bundle uses the unchanged text tower, so text queries match exactly, but documents you add later with the bundle will sit slightly off the BF16 documents.

License and attribution

Apache License 2.0, inherited from the base model. A copy is included as LICENSE.

Modifications made to the base model: the vision and audio towers' weight matrices were quantized to 4 bits (RTN, asymmetric, group size 64 along the input dimension, fp16 scale and zero-point), except the audio tower's output projection and the audio embedding projection, which stay BF16. The text tower is unchanged. No weights were trained or fine-tuned.

Base model © Google DeepMind. This is an independently quantized derivative and is not endorsed by Google.