clip-vit-base-patch32

This repository hosts the clip-vit-base-patch32 models exported for the React Native ExecuTorch library as ExecuTorch .pte programs, ready to run on device.

Upstream model: clip-vit-base-patch32

Variants

XNNPACK int8

The xnnpack/*_int8.pte towers are 8da8w (int8 dynamic activations, int8 per-channel weights) on the Linear layers; the text tower also has an int8 token embedding table. ImageNetV2 zero-shot top-1 (1000 images, one per class, "a photo of a {class}."):

Repository structure

config.json                                           43 B
coreml/clip_vit_base_patch32_image_coreml_fp16.pte    168 MB
coreml/clip_vit_base_patch32_text_coreml_fp16.pte     122 MB
coreml/clip_vit_base_patch32_text_coreml_int8.pte     61.0 MB
coreml/config.json                                    1.7 kB
mlx/clip_vit_base_patch32_image_mlx_int8.pte          93.7 MB
mlx/config.json                                       840 B
tokenizer.json                                        2.1 MB
vulkan/clip_vit_base_patch32_image_vulkan_fp16.pte    168 MB
vulkan/clip_vit_base_patch32_text_vulkan_fp16.pte     121 MB
vulkan/config.json                                    1.7 kB
xnnpack/clip_vit_base_patch32_image_xnnpack_fp32.pte  335 MB
xnnpack/clip_vit_base_patch32_image_xnnpack_int8.pte  91.5 MB
xnnpack/clip_vit_base_patch32_text_xnnpack_fp32.pte   242 MB
xnnpack/clip_vit_base_patch32_text_xnnpack_int8.pte   61.4 MB
xnnpack/config.json                                   2.9 kB

Compatibility

These files are published for the ExecuTorch v1.4.1 runtime. ExecuTorch gives no forward compatibility guarantee, so an older runtime may fail to load them.

To use them in React Native ExecuTorch, pass the model constant shipped in the library's model registry to the corresponding task pipeline. See the documentation.

To load these files in your own ExecuTorch runtime, read the compatibility note first.