EmbeddingGemma 2 (text and vision), quantized for MLX

This is a quantized copy of Google's EmbeddingGemma 2, made for on-device search in the Einso app (https://einso.app).

The model is Google's work and is released under the Apache License 2.0 (included here as LICENSE). This copy is shared under the same licence.

What was changed

• The audio tower (audio_tower.*, embed_audio.*) was left out. The text and vision towers are unchanged in structure.
• Weights were quantized with MLX's affine quantization (group size 64, scales and biases stored beside each weight): the text and vision layers' linear weights at 6 bits, the word embedding table at 4 bits. Each layer's bits can be read from its weight and scales shapes. Norms and other small tensors keep Google's original bfloat16 values.
• config.json is Google's, with a quantization entry added. The tokenizer files are Google's, unchanged.

Quality

We compared this copy with Google's bfloat16 weights on 40 board-style notes, 24 search questions (including one in Spanish) and 5 photos:

Use

Queries use the prefix task: search result | query: and documents use title: none | text: , as in Google's model card. Output vectors are mean-pooled and L2-normalised, 768 dimensions.