clip_vit_b32_image — ExecuTorch
- Source: openai/clip-vit-base-patch32
- License: MIT
- Input: [[1, 3, 224, 224]] — RGB, CLIP norm (mean .481/.458/.408, std .269/.261/.276), 224x224
- Output: image embedding [1,512] (unnormalized; L2-normalize before cosine)
Variants
All variants take and return fp32 tensors — swap the .pte file, keep your app code.
| build | file | size (MB) | parity vs fp32 eager (worst corr) | Mac median (ms)* |
|---|---|---|---|---|
| fp32 | clip_vit_b32_image_xnnpack_fp32.pte |
351.6 | 1.000000 | 19.0 |
| fp16 | clip_vit_b32_image_xnnpack_fp16.pte |
180.7 | 0.999996 | 26.1 |
| Core ML (fp16, iOS) | clip_vit_b32_image_coreml_all.pte |
176.2 | 0.999998 | 3.5 |
The Core ML build is the same graph lowered to Apple's Neural Engine instead of XNNPACK, which is CPU-only. Measured on an iPhone 17 Pro across seven models, it runs 3.5x to 13.9x faster (median 12x) at roughly half the file size — for example Depth-Anything-V2-Small at 500.8 ms against 42.7 ms, and MODNet at 81.7 ms against 5.9 ms. It computes in fp16 and is iOS-only; the XNNPACK files stay the portable option and are what runs on Android.
*Mac arm64, single process, median of 10 — a reference point for relative cost only, not a device number (torch eager fp32 on the same machine: 18.5 ms).
Checked in the task's own units
Correlation is a first filter, and on an embedding it is a weak one: a build can read 0.999 against eager and still move an answer, because what decides is whether the error is smaller than the gap between the answer a query gets and the runner-up.
The test is near-duplicate retrieval over 52 photographs, each also present centre-cropped to 90% and re-encoded as JPEG. Every item has one obviously correct nearest neighbour, and each build has to find it.
| build | correct neighbour kept | worst score shift | budget spent |
|---|---|---|---|
| fp32 | 104 of 104 | 0.0000 | 0% |
| fp16 | 104 of 104 | 0.0023 | 32% |
| Core ML (fp16, iOS) | 104 of 104 | 0.0030 | 43% |
The closest decision eager makes on this set is 0.0071, and the bar is half of it. An earlier version of this card cleared int8 on "cosine similarity of the embeddings", which is correlation wearing a task metric's clothes — it never asked whether a retrieval answer moved.
Withdrawn: int8 (2026-08-27)
clip_vit_b32_image_xnnpack_int8.pte was published and has been withdrawn. It was
cleared on "cosine similarity of the image embeddings, median 0.9988 over 10 real images",
which never asked whether a retrieval answer moved.
Re-measured on the test above it still matches all 104 copies to their originals, but it moves a pair score by 0.0404 against a closest decision of 0.0071 — 567% of the room available, against a bar of 50%. Its zero-shot label also moved on one image of the 104, where no other build moved any. And the number is generous: this shelf calibrated the int8 build on these same photographs.
At 95.9 MB it was the smallest build and size was the reason to reach for it. fp16 is 180.7 MB and spends 32% of the same budget; the Core ML build is 176.2 MB, spends 43%, and runs 5x faster.
Verification (executorch 1.4.0, torch 2.13.0)
Parity is measured against the fp32 eager model on real image input; corr is
the correlation over all elements of each output tensor.
| output | shape | max_abs_diff | corr |
|---|---|---|---|
| 0 | [1, 512] | 9.179e-06 | 1.000000 |
XNNPACK delegate coverage (fp32): 69.3% (390/563 ops); ops left on the portable kernels: aten.expand_copy.default x49, aten.native_layer_norm.default x26, aten.mul.Scalar x24, aten.logical_not.default x24, aten.eq.Scalar x12, aten.full_like.default x12, aten.any.dim x12, aten.where.self x12, aten.embedding.default x1, aten.select_copy.int x1
Conversion
torch.export -> to_edge_transform_and_lower(partitioner) -> .pte (conversion scripts: executorch-models)
This repo holds both towers: clip_vit_b32_image_xnnpack_fp32.pte (image) and
clip_vit_b32_text_xnnpack_fp32.pte (text, fixed len 77 + attention mask).
L2-normalize both embeddings, then cosine-match.