EmbeddingGemma 2 for Core ML

This is google/embeddinggemma-2 (Google DeepMind's text and image embedding model, one shared 768-d space) converted to multifunction Core ML models that run on the Apple Neural Engine, one per image budget. Every stage of the conversion was checked against the original PyTorch model. It is an unofficial conversion. It is not made or endorsed by Google.

• Base model: google/embeddinggemma-2 at revision 914f7f89142e33e77833254d9c9b90c3cef7303b (model.safetensors sha256 197a32965d4b1105faf060417baa899e193fb73cd401f42ec9295234d5553d79).
• Modalities: text and images. Audio and video are not converted.
• Variant shipped: int8_table4: int8 per-channel conv weights, int4 per-block (32) table; tables and norms FP16.
• Layout: one folder per image budget, b70/ and b280/. Each holds the text functions and that budget's image functions; a device uses one budget only, so it downloads one folder's archive.
• Size: apps download one file, b<budget>/EmbeddingGemma2.aar (an LZFSE Apple Archive of the package and the tokenizer: 343.2 MB for budget 70, 352.9 MB for budget 280; the s4 gate is 360 MB for the larger), described by b<budget>/manifest.json (size, SHA-256 and image budget). Unpacked it is 435.0 / 454.9 MB; the package alone is 402.8 / 422.8 MB. The same files are also in each folder unpacked, each listed with its size and SHA-256 in b<budget>/files.json.
• Runs on: iOS 18+ and macOS 15+ (the model uses ios18 ops). It is built for .cpuAndNeuralEngine.
• Swift package: EmbeddingGemma (download, verification, tokenizer, image preprocessing, plus an on-disk vector index). See "Use from Swift" below.

License and terms

• The weights in this repo are a modified version of google/embeddinggemma-2, under the Apache License, Version 2.0. This is the license the upstream model card declares at the pinned revision (license: apache-2.0, linking Google's Gemma 4 license, which is the Apache 2.0 text). The original model is by Google DeepMind. See LICENSE (the full Apache 2.0 text) and NOTICE.
• Changes made (Apache 2.0, section 4(b)): the original weights were re-implemented in PyTorch, converted to Core ML, cast to FP16, and compressed (int8 per-channel convolution weights, int4 per-block embedding table). The vectors are close to the original's but not identical (see "Accuracy").
• Upstream use policy, passed through: the upstream model card says "Deployments must adhere to the Gemma Prohibited Use Policy." That requirement comes from the upstream model, and it applies to this conversion too. The upstream card also says developers are responsible for application-level safeguards (retrieval filtering, fairness testing) suited to their use case. Read Google's model card for its intended uses, limitations and ethical considerations. They all apply here.
• The tokenizer files (tokenizer/tokenizer.json, tokenizer/tokenizer_config.json) are unmodified copies from the upstream revision above.
• The conversion code and the Swift package are Apache-2.0 too (the GitHub repo's LICENSE and NOTICE).

Functions

One .mlpackage per image budget, b70/EmbeddingGemma2.mlpackage and b280/EmbeddingGemma2.mlpackage, each with nine functions with fixed shapes: the four text functions (the same in both) and the five image functions of its budget. Both were split from one 14-function package, and every function gives the same output bytes as there. Every function returns embedding, float32 [1, 768], L2-normalized.

Image shapes (H x W). The bucket comes from the aspect ratio r = width / height: tall r < 0.6, portrait r < 0.83, square r <= 1.2, landscape r <= 1.67, wide above that.

• Text is <bos> + tokens(prompt + text) + <eos>, padded with <pad> (id 0) to the smallest length that fits. Longer inputs are cut to 128 tokens, keeping <bos> and <eos> (the upstream model allows up to 8K tokens). Use the upstream prompts: task: search result | query: for queries, title: none | text: for documents (also task: classification | query: and task: clustering | query: ).
• Images are resized straight to the bucket size (no crop or padding). Use 8-bit sRGB, antialiased bicubic resampling, and scale by 1/255 with no mean or std normalization, as the reference processor does. The Swift package does this.
• Matryoshka: you can truncate the 768-d output to 512, 256 or 128 dims and re-normalize. The Swift package defaults to 256. All accuracy numbers below are at 768 dims.

Accuracy against the original model

Measured by the pipeline on 200 hand-written texts (120 of them search queries) and 200 images (CC0 and public domain photos from Wikimedia Commons, plus synthetic screenshots, memes and receipts; 14 categories), at both image budgets. Cosine values are the minimum over all inputs, not the mean.

Retrieval with the shipped variant (the 120 queries ranking the 200 images, budget 70 / 280):

For 98.3% of queries (worst budget), the exact-aspect model's top-1 image is in the shipped model's top 10. Image-to-image precision@10 by category label is at most 0.012 below the exact-aspect FP model. Swift tokenization matches the reference ids for every fixture text (True). The 99th-percentile pixel difference against the reference preprocessing is 0.000000 (0-1 units).

Caveats:

• The retrieval gates use a bucketed FP reference. The upstream processor resizes each image to its own aspect-preserving size. This model has five fixed shapes per budget, so even with no compression, the bucketed vector differs from the exact-aspect one (min cosine 0.916 at budget 70, 0.893 at 280). That is why recall@10 against the exact-aspect model is lower than recall@10 against the bucketed reference. The compression and FP16 gates (min cosine >= 0.99, recall@10 >= 0.95, Spearman >= 0.9) are therefore measured against the FP model on the same bucketed images. Bucketing gets its own task-level gates (top-1 in top 10

  = 0.95, category precision@10 drop <= 0.03). Measured by category labels, bucketing costs no retrieval quality. Details: ADR-0007 in the GitHub repo.

• Two images fall below the FP16 bar on the Neural Engine. In FP16 on the Neural Engine, fixture images i141 and i161 reach 0.99404 / 0.99405 cosine. The other 398 are at 0.998 or above. On the GPU these two images pass, but the GPU is 2.8x slower at budget 70. They change the top 10 of only a few queries: with only those two vectors swapped in, recall@10 stays at 0.9975 (budget 70) and 0.9992 (budget 280). They were accepted at task level, and images stay on the Neural Engine. The lowest image cosines of the shipped variant are i161 0.99133 (embed_image_280_landscape), i141 0.99322 (embed_image_70_wide), i165 0.99336 (embed_image_280_portrait). Details: ADR-0006.
• The test set is small: 200 texts, 200 images. Multilingual text, code and long documents were not tested separately. For quality on standard benchmarks, see Google's model card. This conversion only claims closeness to the original.

Speed and memory

Mac (s4, CPU_AND_NE, median of 20 runs after warm-up). These were measured on the build Mac; the v1.0.0 build Mac is an M1 Pro with macOS 26. embed_text_16 takes 4.4 ms and embed_image_70_square takes 43.8 ms. In every function, at least 99.81% of the ops are placed on the Neural Engine (MLComputePlan, minimum over the 14 functions, measured on the package the two budget packages were split from).

iPhone (s6, PhotoSearchDemo's benchmark: each iPhone runs its own budget, ImageBudget.auto, 280 on A17 Pro and newer and 70 otherwise: the newest 1,000 photos at 70 or 200 at 280, and 20 queries; times cover preprocessing + prediction):

Worst device: 64.0 ms per image at 70, not measured (no A17 Pro or newer iPhone ran s6 yet) at 280, 6.7 ms per query, peak 630 MB (the gate is 800 MB; the target for budget 70 is 60 ms, which is recorded, not gated).

Storage: the Swift package downloads the archive of the device's image budget, b<budget>/EmbeddingGemma2.aar (343.2 MB at 70, 352.9 MB at 280, resumable), checks its size and SHA-256 against b<budget>/manifest.json, unpacks it (435.0 / 454.9 MB) with the AppleArchive framework and compiles the package. It needs free space for the archive, the unpacked files and the compiled model while it installs; afterwards it keeps the compiled model and the tokenizer. The s4 gate measures the larger archive, 352.9 MB (limit 360 MB).

Use from Swift

// Package.swift: .package(url: "https://github.com/Delavalom/embeddinggemma-coreml", from: "1.0.0")
import EmbeddingGemma
import EmbeddingIndex

let model = try await EmbeddingGemma.load(.huggingFace(revision: EmbeddingGemma.defaultRevision))
let photo = try await model.embed(image: cgImage, orientation: .up)  // [Float], 256-d by default, normalized
let query = try await model.embed(query: "receipts from last week")

The package pins this repo (delavalom/embeddinggemma-2-coreml) at a commit and the SHA-256 of each budget's manifest.json, which pins its archive's size and SHA-256. budget: .auto picks 280 tokens on A17 Pro and newer iPhones and on Macs, and 70 on other devices, and loads that budget's folder (b280/ or b70/) only.

Use from Python (coremltools, macOS 15+)

import numpy as np, coremltools as ct
from tokenizers import Tokenizer
from torchvision.transforms.functional import pil_to_tensor, resize, InterpolationMode
from PIL import Image

BUDGET = 280  # the folder you downloaded: b280/ or b70/; its text functions are the same in both
PKG = f"b{BUDGET}/EmbeddingGemma2.mlpackage"
tok = Tokenizer.from_file(f"b{BUDGET}/tokenizer/tokenizer.json")
# self-check: the layout the model was traced with is <bos> ... <eos>
assert tok.encode("task: search result | query: screenshots of tweets").ids == \
    [2, 8071, 236787, 3927, 1354, 1109, 7609, 236787, 75490, 529, 46331, 1]

def fn(name):
    return ct.models.MLModel(PKG, function_name=name, compute_units=ct.ComputeUnit.CPU_AND_NE)

def embed_text(text, prompt="task: search result | query: "):
    ids = tok.encode(prompt + text).ids
    ids = ids if len(ids) <= 128 else ids[:127] + [ids[-1]]
    n = next(L for L in (16, 32, 64, 128) if L >= len(ids))
    x = np.zeros((1, n), np.int32); x[0, :len(ids)] = ids
    m = np.zeros((1, n), np.int32); m[0, :len(ids)] = 1
    return fn(f"embed_text_{n}").predict({"input_ids": x, "attention_mask": m})["embedding"][0]

SHAPES = {70: {"tall": (576, 240), "portrait": (480, 336), "square": (384, 384), "landscape": (336, 480),
               "wide": (240, 576)},
          280: {"tall": (1152, 528), "portrait": (960, 672), "square": (768, 768), "landscape": (672, 960),
                "wide": (528, 1152)}}

def bucket(r):  # r = width / height
    return ("tall" if r < 0.6 else "portrait" if r < 0.83 else "square" if r <= 1.2
            else "landscape" if r <= 1.67 else "wide")

def embed_image(path):
    img = Image.open(path).convert("RGB")  # like the reference: 8-bit RGB, ICC profile ignored
    name = bucket(img.width / img.height)
    px = resize(pil_to_tensor(img), list(SHAPES[BUDGET][name]), InterpolationMode.BICUBIC, antialias=True)
    px = px.float() / 255
    return fn(f"embed_image_{BUDGET}_{name}").predict({"pixels": px[None].numpy()})["embedding"][0]

def truncate(v, dims=256):  # Matryoshka
    v = v[:dims]; return v / np.linalg.norm(v)

The model is compiled each time MLModel is created. To make many calls, keep one instance per function. The Python path is not gated; the gated path is the Swift package (s5).

Reproducibility

The model was built by an 8-stage pipeline in the GitHub repo. Each stage writes artifacts/<stage>/report.json, which is checked against gates.toml. A stage counts only if its report passed and is current. Each report records git_sha and code_hash (sha256 over the stage's code, the lock file and its gate table), and inputs_sha256 chains it to the reports it read. The reports behind this release:

Reference environment: transformers 5.20.2, sentence-transformers 6.2.0; conversion: coremltools 9.0. To rebuild: just setup, then just stage 0 through just stage 5 (just stage 6 needs an iPhone). Numbers measured on the CPU and the Neural Engine can vary slightly from run to run.

Results

Generated by python -m scripts.gen_results from artifacts/*/report.json and docs/results/*.json. Do not edit by hand.

Model: google/embeddinggemma-2 at revision 914f7f89142e33e77833254d9c9b90c3cef7303b.

Gates

s4 variants

Chosen: int8_table4 (winner: int8_table4). Variants are tried smallest first; the first that passes every gate wins, so later ones may not have run. Download limit: 360 MB.

Shipped per image budget (download MB = the device archive; max_download_mb is the larger): budget 70 EmbeddingGemma2-b70.aar 343.178 MB (9 functions), budget 280 EmbeddingGemma2-b280.aar 352.911 MB (9 functions). Same outputs as the 14-function package (CPU_ONLY, bitwise): True.

s5 Swift parity

Pixels (0-1 units; PNG and synthetic are gated, JPEG includes the decoder and is recorded only, ADR-0004):

Against s0 (recorded only; on JPEGs it includes the decoder, ADR-0008):

Device benchmarks (s6)

Limitations

• Text and images only. Text is limited to 128 tokens; images are limited to budgets of 70 or 280 tokens and five fixed aspect buckets each.
• Results are close to the original model, not bit-identical (see "Accuracy"). Do not mix vectors from this model with vectors from the PyTorch model or another conversion in the same index. The Swift index stores the model revision for that reason.
• Intel Macs have no Neural Engine; they run on the CPU/GPU and are slower. The fastest settings are on the Neural Engine with .cpuAndNeuralEngine.
• The first load compiles the model on the device, which takes a while. The Swift package caches the compiled model.
• Everything in the upstream card's "Limitations" and "Ethical Considerations and Risks" applies.

Citation

Please cite the original model as Google DeepMind's model card asks (google/embeddinggemma-2). For this conversion:

@software{embeddinggemma_coreml,
  title  = {EmbeddingGemma 2 for Core ML},
  author = {Delavalom},
  year   = {2026},
  url    = {https://huggingface.co/delavalom/embeddinggemma-2-coreml},
  note   = {Core ML conversion of google/embeddinggemma-2}
}