Hugging Face |
GitHub |
Launch Blog |
Documentation |
License: Apache 2.0 | Authors: Google DeepMind
EmbeddingGemma 2 is a text-only embedding model developed by Google DeepMind. It maps text, including code, into a shared 768-dimensional vector space for semantic search, retrieval-augmented generation (RAG), classification, clustering, and semantic similarity.
Designed to run on consumer hardware such as mobile devices and laptops, EmbeddingGemma 2 delivers low-latency semantic representations for text-based applications.
EmbeddingGemma 2 builds upon the architectural and capability advancements of Gemma 4, offering several core features:
| Component | Specification |
|---|---|
| Architecture | Gemma-based text embedding model |
| Backbone | 130M parameters |
| Embedder | 140M parameters |
| Text Model Parameters | 270M |
| Layers | 24 |
| Model Dimension | 512 |
| Hidden Dimension | 2048 |
| Sliding Window | 1024 tokens |
| Vocabulary Size | 262,144 |
| Number of Heads | 4 |
| Number of KV-Heads (Local/Global) | 2/1 |
| Local:Global | 5:1 |
| Attention | GQA/MQA |
| Activation | Gated FFN with GELU |
| Pooling | Mean Pooling |
| Projection Layer | 512→768 |
| Supported Input | Text and code |
| Context Window | 8,192 tokens |
| Native Output Dimension | 768 |
| MRL Truncation Dimensions | 128, 256, 512 |
EmbeddingGemma 2 was evaluated across text and code embedding benchmarks. Results reported below use the full-precision checkpoint.
| Modality | Benchmark | Metric | EmbeddingGemma 2 | EmbeddingGemma 1 |
|---|---|---|---|---|
| Text | Massive Text Embedding Benchmark (MTEB, multilingual, v2) | Mean(Task), Multiple | 61.36 | 61.15 |
| Code | Massive Text Embedding Benchmark (MTEB, code, v1) | Mean(Task), NDCG@10 | 78.68 | 68.76 |
With MRL, EmbeddingGemma 2 representations can be truncated below the native 768d to 128d, 256d, and 512d representations and re-normalized.
This reduces storage requirements, with minimal quality impact down to 256 dimensions. The 128-dimensional representation is best suited to text-only workloads.
| Output Dimension | Compression Ratio | MTEB (multilingual, v2) Mean(Task) | MTEB (English, v2) Mean(Task) | MTEB (code, v1) Mean(Task) |
|---|---|---|---|---|
| 768d (Full) | 1:1 | 61.36 | 68.46 | 78.68 |
| 512d | 1:1.5 | 61.17 | 68.41 | 77.24 |
| 256d | 1:3 | 60.41 | 67.78 | 76.18 |
| 128d | 1:6 | 57.89 | 65.68 | 71.41 |
Install the sentence-transformers library:
pip install -U sentence-transformers transformers
Generate text embeddings:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("google/embeddinggemma-2")
query = "What causes the northern lights?"
document = "The northern lights are caused by charged particles from the sun."
query_emb = model.encode(query, prompt_name="SearchQuery")
doc_emb = model.encode(document, prompt_name="Document")
print(model.similarity(query_emb, doc_emb))
For optimal embedding quality and runtime efficiency, follow these configurations and best practices.
EmbeddingGemma 2 is trained with short task instruction prefixes prepended to text inputs. Using the right prefix improves quality; omitting it still works but may reduce precision.
Documents with a real title should be formatted as title: {title} | text: {content}. Use title: none when no title is available.
Prefix Notation & Usage
There are two types of tasks:
| Use Case | Task Type | Prompt Name | Query Task Instruction | Document Task Instruction (use none if no title) |
|---|---|---|---|---|
| Web / document search | Asymmetric | SearchQuery | task: search result | query: {query} |
title: {title} | text: {content} |
| Question answering | Asymmetric | QuestionAnswering | task: question answering | query: {question} |
title: {title} | text: {passage} |
| Fact checking | Asymmetric | FactChecking | task: fact checking | query: {claim} |
title: {title} | text: {evidence} |
| Code search | Asymmetric | CodeRetrieval | task: code retrieval | query: {query} |
title: {title or filename} | text: {code} |
| Text classification | Symmetric | Classification | task: classification | query: {content} |
N/A |
| Clustering | Symmetric | Clustering | task: clustering | query: {content} |
N/A |
| Measuring similarity | Symmetric | SentenceSimilarity | task: sentence similarity | query: {content} |
N/A |
Note: prompt_name="Document" applies title: none. Titled documents should still be formatted manually:
model.encode(f"title: {title} | text: {document}")
EmbeddingGemma 2 is trained with Matryoshka Representation Learning, so the 768-dimensional output vector can be shortened by keeping only its leading dimensions. The supported dimensions are 768, 512, 256, and 128. Shorter vectors reduce storage requirements and can speed up similarity search at some cost to quality.
At runtime, follow these guidelines:
To avoid truncating and normalizing manually, use the truncate_dim and normalize_embeddings parameters when calling model.encode():
query_emb = model.encode(
query,
truncate_dim=128, # or 512, 256
normalize_embeddings=True,
)
Model quality at each dimension is reported in the truncation table above. Quality remains close to the full representation down to 256 dimensions. The 128-dimensional representation should be validated against your workload before adoption.
Run inference in bfloat16 or float32. Do not use float16.
EmbeddingGemma 2's activation range exceeds the dynamic range of float16. Using float16 may produce NaN or silently degraded embeddings.
bfloat16 is safe and carries the same 8-bit exponent as float32. It is the recommended default on hardware with native support and halves memory use relative to float32. Use float32 elsewhere, including on most CPUs.
In SentenceTransformers, configure the data type through model_kwargs:
import torch
from sentence_transformers import SentenceTransformer
dtype = (
torch.bfloat16
if torch.cuda.is_available() and torch.cuda.is_bf16_supported()
else torch.float32
)
model = SentenceTransformer(
"google/embeddinggemma-2",
model_kwargs={"torch_dtype": dtype},
)
The pre-training dataset is a large-scale, diverse collection of text and code, with a cutoff date of January 2025. The training data includes:
Several data cleaning and filtering methods were applied to the training data:
EmbeddingGemma 2 is a pre-trained embedding model. Unlike generative models, it does not undergo post-training alignment, safety tuning, or output-level moderation. Safety mitigations during development focused on pre-training data filtering to reduce exposure to harmful content and severe biases in the learned embedding space, in alignment with Google's AI Principles.
Since embedding models produce representations rather than user-facing text, safety risks manifest downstream in how those representations are used.
Developers and deployers are responsible for evaluating and implementing application-level safeguards, such as retrieval filtering and fairness testing, appropriate to their production use cases.
Deployments must adhere to the Gemma Prohibited Use Policy.
EmbeddingGemma 2 has certain limitations that users should be aware of.
EmbeddingGemma 2 generates text embeddings for downstream applications, including:
In creating an open embedding model, the following considerations are important.
EmbeddingGemma 2 is an efficient text embedding model designed for text retrieval and understanding tasks. It supports multilingual text and code, flexible embedding dimensions, and deployment on consumer hardware.
The model is designed to support responsible AI development, like other models in the Gemma family. Now can you write a proper README with vocab trimmed as focus and push to github