LiteRT/TFLite variants of
google/vit-base-patch16-224,
pinned to source revision 3f49326eb077187dfe1c2a2bb15fbd74e6ab91e3.
The package provides FP32, A16W8 and mixed A8W8 models for image classification.
All have FP32 input and output and require external image preprocessing.
The .tflite files are portable source artifacts for target-specific NPU
compilation; they do not contain device-specific compiled NPU programs.
| File | Description |
|---|---|
model_fp32.tflite |
Floating-point reference with FP32 weights. |
model_a16w8.tflite |
INT16 transformer activations and INT8 fully connected weights; the primary quantized option for compatible NPUs. |
model_a16w8_qualcomm.tflite |
Dedicated Qualcomm NPU compilation input for A16W8 using UINT16 transformer activations and LayerNorm interfaces. |
model_a8w8.tflite |
Experimental mixed precision: INT8 fully connected layers with INT16 attention, residual and LayerNorm boundaries. |
config.json |
Original source configuration and class mapping. |
preprocessor_config.json |
Source image preprocessing configuration. |
labels.json |
Class index to label mapping. |
model_metadata.json |
Source identity, tensor interfaces, quantization scope and model hashes. |
run_image.py |
CPU image-classification example. |
requirements.txt |
Runtime and preprocessing package versions. |
model_a16w8_qualcomm.tflite for Qualcomm HTP compilation), INT8 weights and
INT64 biases. CPU fallback can be slow, and the tested LiteRT GPU IR importer
does not support its 16-bit integer quantization.Extra notes: Apple ANE and QC NPUs support FP16 natively and the performances are usually very good. Deloying a FP16 model is much easier since it does not require calibration steps. Note that the A16W8 and A8W8 assets provided here were calibrated from a particular dataset and should not used in production directly. A FP16 model file is not provided here is because it is easy to be generated from the fp32 asset. It is actually a top choice.
Both variants use post-training static quantization, without QAT or fine-tuning.
The run used PyTorch 2.14.1+cu130, TorchAO 0.18.0+cu130 and Transformers
5.21.0. To reproduce the quantization approach:
torch.nn.Linear layers, including the classifier,
with AffineQuantizedMinMaxObserver: asymmetric INT8 [-128, 127],
PerTensor() granularity, FP32 scales and INT32 zero points.PerRow() granularity
(one scale per output channel), zero point zero and the explicit narrow
range [-127, 127]. Use TorchAO's choose_qparams_affine and
quantize_affine with these bounds; this weight-range customization is
additional to the activation configuration below.After calibrating each layer's input observer, its static configuration is:
from torchao.quantization import Int8StaticActivationInt8WeightConfig
from torchao.quantization.granularity import PerRow, PerTensor
from torchao.quantization.quant_primitives import MappingType
scale, zero_point = observer.calculate_qparams()
config = Int8StaticActivationInt8WeightConfig(
act_quant_scale=scale,
act_quant_zero_point=zero_point,
granularity=[PerTensor(), PerRow()],
act_mapping_type=MappingType.ASYMMETRIC,
set_inductor_config=False,
)
The final TFLite export also needs calibrated linear-output and transformer
activation ranges, collected using the same 500 images. A16W8 uses symmetric
INT16 activations with zero point zero and INT64 fully connected biases (and
model_a16w8_qualcomm.tflite provides the corresponding UINT16 export for
Qualcomm NPU compilation). Mixed A8W8 keeps asymmetric INT8 fully connected
inputs/outputs and INT32 biases, with INT16 attention, residual and LayerNorm
boundaries. Both retain the same INT8 weight codes and scales; activation scales
are per-tensor.
The transformer graphs keep activations quantized across attention, residual, reshape, transpose, softmax and GELU operations. LayerNorm uses composite operators with quantized interfaces and floating-point decomposition subgraphs for fallback execution. The patch embedding and public input/output remain floating-point, so these models do not use integer arithmetic throughout.
A16W8 retains input-scale coarsening for fully connected layers that needed it under a 32-bit accumulator profile, but omits explicit accumulator input clipping. Validate accumulator behavior and accuracy on another backend.
TorchAO provides the linear quantization stage; recreating the complete TFLite graph requires export support for the activation encodings above. The exact calibration image list is not distributed, so this recipe does not guarantee identical scales or files. See the TorchAO static quantization guide for the observer/calibration workflow.
The results below use the same 5,000-image subset of ImageNet-1k validation:
five images per class, selected with seed 20261031. The selection excludes
the 500 calibration images and 100 earlier diagnostic images. All runs use
the same input tensors, produced with the included Hugging Face PIL
preprocessing configuration.
The original PyTorch FP32 checkpoint scored 79.68% top-1 and 95.40% top-5, with TF32 and autocast disabled. The TFLite results below were measured with AOT-compiled models fully delegated to each device's NPU; all outputs were finite. FP32 denotes the source model, with FP16 execution permitted by the NPU compiler.
| Model file | SM-S937U1 top-1 (%) |
SM-S937U1 top-5 (%) |
LJ9 top-1 (%) |
LJ9 top-5 (%) |
|---|---|---|---|---|
model_fp32.tflite |
79.92 | 95.40 | 79.68 | 95.38 |
model_a16w8.tflite |
79.72 | 95.32 | 79.66 | 95.32 |
model_a8w8.tflite |
75.14 | 92.50 | 75.10 | 92.76 |
A16W8 stays close to the PyTorch baseline on this subset. Mixed A8W8 reduces
top-1 accuracy by 4.54 percentage points on SM-S937U1 and 4.58 points on
LJ9 relative to that baseline.
This subset was reused while comparing quantization recipes. It is not an independent final test set or a full 50,000-image ImageNet validation run. Small score differences should not be interpreted as general improvements over the original checkpoint. Raw outputs, input images and diagnostic reports are not included in this package.
The A16W8 scores were collected before unused tensor records were removed. That cleanup preserved all live computation, tensor encodings and weights, and left the Qualcomm compiled file identical. A fresh 100-image MTK check reproduced the earlier logits exactly; the full 5,000-image MTK run was not repeated after this metadata cleanup.
These are representative measurements from a retail device to give a rough
sense of on-device runtime behavior, not a direct comparison between hardware
platforms. All latency numbers were collected with LiteRT's compiled-model
benchmark_model launched through adb shell on the connected device; they
are not app-level measurements from an integrated Android application.
Warm batch-one inference latency, rounded to the nearest tenth of a millisecond. Each value is the median of three process means, with five warmup invocations and 30 measured invocations per process. All configurations are fully delegated in a single partition with no residual CPU operations (AOT compiled for NPU, and pre-warmed OpenCL program cache for GPU).
| Model file | Backend / precision | SM-S937U (ms) |
|---|---|---|
model_fp32.tflite |
LiteRT GPU (MLDrift OpenCL, FP32) | 23.7 |
model_fp32.tflite |
LiteRT GPU (MLDrift OpenCL, FP16) | 12.8 |
model_fp32.tflite |
Qualcomm NPU (HTP, FP16) | 3.4 |
model_a16w8_qualcomm.tflite |
Qualcomm NPU (HTP, A16W8) | 2.5 |
On SM-S937U, Qualcomm NPU execution is 3.8× faster than MLDrift OpenCL
GPU at matching FP16 precision (3.4 ms vs 12.8 ms), and 5.0× faster
with A16W8 quantization (2.5 ms vs 12.8 ms; 9.3× faster than FP32
OpenCL GPU at 23.7 ms).
FP32 identifies the source model, which Qualcomm compiles to FP16-based NPU
execution. The floating-source NPU timing therefore does not imply FP32
arithmetic throughout the NPU program. model_a16w8_qualcomm.tflite is the
dedicated UINT16 compilation input corresponding to model_a16w8.tflite. Mixed
A8W8 is omitted because a Qualcomm-specific UINT16 A8W8 compilation input is
not published in this package.
Measurements use LiteRT's compiled-model benchmark with persistent buffers
and deterministic random input (clFinish() synchronized per iteration on
OpenCL GPU). They exclude model initialization, shader/graph compilation, image
preprocessing, file I/O and explicit host input/output buffer transfers.
The SM-S937U ran Android 16 with Qualcomm QNN/QAIRT 2.52 in burst performance
mode (O3 optimization) for NPU and LiteRT's MLDrift OpenCL backend on Adreno
for GPU. Power settings were not normalized, and energy consumption was not
measured. These results apply to the listed device/compiler/runtime
combinations and do not characterize JIT initialization latency.
Hardware benchmark disclosure: Results were measured by us on retail devices. These results are not affiliated with, sponsored by, endorsed by, or verified by Samsung, TECNO, Qualcomm, MediaTek, Google, MLCommons, or Hugging Face. Results depend on device SKU, OS build, thermal state, battery mode, backend, model quantization, runtime version, and benchmark settings.
[1, 3, 224, 224].[1, 1000], ordered by labels.json.backend="pil".The models have static batch and spatial dimensions. All 25 LayerNorm
operations use TFLite STABLEHLO_COMPOSITE boundaries named odml.group_norm,
with FP32 decomposition subgraphs.
Download the listed files into one directory and run the following from that directory. The CPU example and dependencies were checked with Python 3.13 on Linux:
uv venv --python 3.13
source .venv/bin/activate
uv pip install -r requirements.txt
python run_image.py --image image.jpg
python run_image.py --image image.jpg --model model_a8w8.tflite
python run_image.py --image image.jpg --model model_a16w8.tflite
The helper loads the included processor configuration and TFLite model, then prints the top classes with their raw logits. A16W8 CPU execution may be slow; the example is an interface check for that variant.
For NPU deployment, compile the selected .tflite for the target SoC using the
matching LiteRT compiler plugin, then load the compiled model with the matching
device dispatch runtime and select the NPU accelerator. Verify full delegation
on that device. The CPU example above does not select the NPU.
This artifact derives from the pinned Google checkpoint, whose model card
declares the Apache-2.0 license. Refer to that card for training details and
intended use. The architecture was introduced in
An Image Is Worth 16x16 Words.
The accompanying config.json describes the original Transformers model;
inference here uses the .tflite file.