litert-community/Phi-4-mini-instruct
This model provides a few variants of microsoft/Phi-4-mini-instruct that are ready for deployment on Android using the LiteRT (fka TFLite) stack, MediaPipe LLM Inference API (in maintenance-only mode) and LiteRT-LM.
Use the models
Colab
Disclaimer: The target deployment surface for the LiteRT models is Android/iOS/Web and the stack has been optimized for performance on these targets. Trying out the system in Colab is an easier way to familiarize yourself with the LiteRT stack, with the caveat that the performance (memory and latency) on Colab could be much worse than on a local device.
Android
Edge Gallery App
Download or build the app from GitHub.
Install the app from Google Play.
Follow the instructions in the app.
LiteRT-LM in your own app
Add com.google.ai.edge.litertlm:litertlm-android to your app as shown in the
LiteRT-LM Kotlin guide, and load
Phi-4-mini-instruct_multi-prefill-seq_q8_ekv4096.litertlm from this repository.
Performance
Android
Note that all benchmark stats are from a Samsung S24 Ultra with 1280 KV cache size with multiple prefill signatures enabled.
| Backend | Quantization scheme | Context length | Prefill (tokens/sec) | Decode (tokens/sec) | Time-to-first-token (sec) | Model size (MB) | Peak RSS Memory (MB) | GPU Memory (MB) |
|---|---|---|---|---|---|---|---|---|
CPU |
dynamic_int8 |
4096 |
66.53 tk/s |
7.28 tk/s |
15.90 s |
3906 MB |
5308 MB |
N/A |
GPU |
dynamic_int8 |
4096 |
314.01 tk/s |
10.39 tk/s |
10.32 s |
3906 MB |
4107 MB |
4608 MB |
- Model Size: measured by the size of the .tflite flatbuffer (serialization format for LiteRT models)
- Memory: indicator of peak RAM usage
- The inference on CPU is accelerated via the LiteRT XNNPACK delegate with 4 threads
- Benchmark is done assuming XNNPACK cache is enabled
- Benchmark is run with cache enabled and initialized. During the first run, the time to first token may differ.
- dynamic_int8: quantized model with int8 weights and float activations.