litert-community/embeddinggemma-300m

Main Model Card: google/embeddinggemma-300m

Overview

This model card provides a few variants of the EmbeddingGemma model that are ready for deployment on Android and iOS using LiteRT, or on Android via the Google AI Edge RAG Library.

Use the models

LiteRT

• Try out the demo example on GitHub.

RAG

• Try out the EmbeddingGemma model in the in the Google AI Edge RAG Library. You can find the SDK on GitHub or follow our Android guide to install directly from Maven. We have also published a sample app.
• Use the sentencepiece model as the tokenizer for the EmbeddingGemma model.

Performance

Android

Note that all benchmark stats are from a Samsung S25 Ultra.

*Mixed Precision refers to per-channel quantization with int4 for embeddings, feedforward, and projection layers, and int8 for attention (e4_a8_f4_p4).

Notes:

• Init time: the cost paid once per application initialization – subsequent inferences do not pay this cost
• Memory: indicator of peak RAM usage
• Model Size: measured by the size of the .tflite flatbuffer (serialization format for LiteRT models)
• The inference on CPU is accelerated via the LiteRT XNNPACK delegate with 4 threads
• Benchmark is run with cache enabled and initialized. During the first run, the latency may differ.

Google Tensor Compilation Guide

To compile for Google Tensor hardware, please follow the step-by-step instructions available in the Google Tensor Official User Guide.

For detailed Google Tensor Compiler configuration, refer to the Install Google Tensor SDK section within the LiteRT_AOT_Compilation_Tutorial.ipynb notebook.

Please note that the embedding-gemma models were compiled using standard configurations; no additional compilation flags are required.