EmbeddingGemma 2 GGUF

Community GGUF quantizations of google/embeddinggemma-2.

☕ If this GGUF made your day easier, a coffee would make mine.
Send a coffee ☕
I build and test these releases myself. Your coffee helps keep me going.
Thank you for supporting this work.

About EmbeddingGemma 2

EmbeddingGemma 2 is a multilingual, multimodal embedding model from Google DeepMind. It maps text and code, images, video, and audio into a shared 768-dimensional vector space and has an 8,192-token shared context window. The upstream checkpoint has a 270M-parameter text path plus optional vision and audio encoders. This release contains a quantized text backbone and a shared BF16 vision/audio projector. Text, image, and audio inputs returned normalized 768-dimensional vectors in local llama-server CPU smoke checks. See the official model card for supported task prefixes, modalities, and input guidance.

Embedding evaluation

Every value below comes from local measurements of the locked BF16 GGUF and quantized files; the numbers are not copied from the upstream model card. The fixed task evaluation uses the 1,379-pair test split of MTEB STSBenchmark STS, dataset revision 96943a16ea6a35129e253c659081cb59daf81b30. It covers 2,552 unique sentences with the task: sentence similarity | query: prefix, an 8,192-token context, and the CPU llama.cpp runtime at commit 9c2e0e491a822adae1f0b1c831adb4160057d24f. Spearman and Pearson measure correlation between cosine similarity and human scores. Mean and 5th-percentile cosine measure vector agreement with the BF16 GGUF reference. Higher values indicate stronger task correlation or closer vector agreement; smaller files use less disk. This is a task-specific embedding evaluation, not the full MTEB benchmark. Token-level KLD, next-token Top-1, PPL, and RMS probability deltas do not apply to this embedding model.

There are 26 quantized text artifacts in this release. Hold-out measurements are available for 24; the two TQ variants passed runtime smoke checks but were not included in this evaluation checkpoint, so no quality results are claimed for them.

BF16 reference baseline

Quantized variants compared with BF16

Q5_K_S and Q4_K_S are the current memory/fidelity candidates on this measured task: their Spearman scores are 0.874892 and 0.873112 with sizes 0.210670 GB and 0.178164 GB. No VRAM-fit or throughput claim is made. Q1_0 and IQ1_S return valid vectors but show substantial loss on this task; the TQ formats have smoke results only. The machine-readable evaluation summary and reproducibility manifest include the measurements and method. The Q8_0 artifact and BF16 vision/audio projector also passed local text, image, and audio embedding smoke checks.

Quick start

Start an embedding endpoint with llama.cpp:

llama-server \
  -m EmbeddingGemma-2-Q8_0.gguf \
  --mmproj mmproj-EmbeddingGemma-2-BF16.gguf \
  --embedding \
  --pooling mean \
  --ctx-size 8192 \
  --port 8080

Send an English search query using the model's task prefix:

curl http://localhost:8080/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"EmbeddingGemma-2","input":"task: search result | query: What causes the northern lights?"}'

For text-only workloads, omit --mmproj. For documents, use title: {title} | text: {content} or title: none | text: {content} when there is no title. Keep query and document vectors at the same dimension. The server returns normalized vectors by default.

Reproducibility and validation

See the reproducibility manifest, artifact measurements, and SHA256 checksums. The manifest records the locked upstream revision and model hash, BF16 GGUF source, converter/runtime revision, model-specific calibration/imatrix, quantization commands, and smoke profile. Raw conversion, imatrix, quantization, and validation logs are kept locally and are not part of this repository.

License and attribution

The upstream model is licensed under Apache 2.0. These are community GGUF quantizations, not an official Google or Google DeepMind release or endorsement.