Smaug-Decisions-LiteRT: Smaug-Decisions on phones
Frontier-level audio decisions, offline on a phone.
3.45 GB on the phone GPU, 59.4 Earshot audio skill: level with Gemini 4 Argon (59.4), within 0.7 of full-precision Smaug-Decisions.
1. Model Introduction
Smaug-Decisions, the 4B decision model that hears, sees and reads, in one 3.45 GB file for Google's on-device runtime LiteRT-LM, with the audio and vision encoders inside, running on the phone GPU. It answers a typed question about audio, an image or text with a probability per option, in one prefill and with no generated text.
It scores 59.4 on Earshot audio decisions, 53.9 on Decision Index text and 55.1 on images (full-precision Smaug-Decisions: 60.1 / 56.6 / 57.2 on the same sets), and runs fully offline in the Cascade app on iPhone and Android next to Smaug-Audio-Decisions-0.7B, which answers the easy audio questions first.
import json
from huggingface_hub import hf_hub_download
manifest = json.load(open(hf_hub_download("abacusai/Smaug-Decisions-LiteRT", "manifest.json")))
path = hf_hub_download("abacusai/Smaug-Decisions-LiteRT", "smaug-decisions-full-vis280.litertlm") # 6 GB phones; vis560 for 8 GB+
2. Files: which device gets which
All files share the same weights; the smaller ones leave out the encoders a device does not need. In the names, vis560 means
images at 560 image tokens (more detail, for phones with 8 GB of RAM or more) and vis280 images at 280 tokens (for 6 GB phones;
an image needs about 1 GB of working memory instead of 2 GB). manifest.json lists each file with its size, sha256, inputs and
download URL.
| file | size | answers | for |
|---|---|---|---|
smaug-decisions-full-vis560.litertlm |
3.45 GB | text, audio, images | phones with 8 GB of RAM or more |
smaug-decisions-full-vis280.litertlm |
3.45 GB | text, audio, images | 6 GB phones |
smaug-decisions-noaudio-vis560.litertlm |
3.15 GB | text, images | 8 GB+ phones where Smaug-Audio-Decisions-0.7B handles all audio |
smaug-decisions-noaudio-vis280.litertlm |
3.15 GB | text, images | the same on 6 GB phones |
smaug-decisions-novision.litertlm |
3.28 GB | text, audio | audio and text, no photos |
smaug-decisions-textonly.litertlm |
2.97 GB | text | text only |
Phones under 6 GB run Smaug-Audio-Decisions-0.7B alone.
In the Cascade app
The iOS and Android apps ship the same two models:
| iOS | Android | |
|---|---|---|
| Smaug-Audio-Decisions-0.7B (audio, inside the app, works with no download) | Core ML, int8 weights, int8 embedding table | ONNX Runtime, int8 weights, the same int8 embedding table |
| Smaug-Decisions (downloaded in the app from this repo) | LiteRT-LM on the phone GPU (Metal) | LiteRT-LM on the phone GPU |
| app size | 602 MB | about 582 MB (Play download, arm64) |
Which Smaug-Decisions file a phone gets:
| phone RAM | default download | file |
|---|---|---|
| 8 GB or more | Full cascade, 3.45 GB | smaug-decisions-full-vis560.litertlm |
| 6 GB | Full cascade, 3.45 GB | smaug-decisions-full-vis280.litertlm |
| under 6 GB | nothing: Smaug-Audio-Decisions-0.7B only | - |
Nothing downloads until the user taps Download, and before that they can switch from the full cascade to text + images (smaug-decisions-noaudio-*,
3.15 GB; Smaug-Audio-Decisions-0.7B answers all audio), audio + text (smaug-decisions-novision, 3.28 GB; no photos) or Smaug-Audio-Decisions-0.7B only
(no download). smaug-decisions-textonly is for other apps; the Cascade app does not offer it. Audio goes to Smaug-Audio-Decisions-0.7B first and
escalates to Smaug-Decisions below 0.9 confidence; text and images go straight to Smaug-Decisions.
3. Model Summary
| Architecture | Gemma 4 E4B (language model + audio and vision encoders), fine-tuned as Smaug-Decisions; letter-logit readout |
| Weights | int4 decoder, int2 embedding tables, int8 audio encoder, an output head that holds only the 20 option letters |
| Input | Text; audio (16 kHz mono); one image at 280 or 560 image tokens; a question with 2 to 20 options |
| Output | The logits of the option letters A..T at the last position, read as a probability per option |
| Context | 2,048 tokens (prompt included) |
| Runtime | LiteRT-LM, GPU backend (Metal on iPhone) or CPU |
| Base Model | abacusai/Smaug-Decisions |
4. Evaluation
The 3.45 GB file keeps almost all of Smaug-Decisions' quality: within a point on audio and its calibration, and two to three points on text and images, at about a fifth of the size.
| Smaug-Decisions(full precision, 16 GB) | Smaug-Decisions-LiteRT(3.45 GB, on the phone) | change | |
|---|---|---|---|
| Earshot audio skill ↑ | 60.1 | 59.4 | −0.7 |
| Earshot calibration error (ECE) ↓ | 0.069 | 0.074 | +0.005 |
| Decision Index text ↑ | 56.6 | 53.9 | −2.7 |
| Decision Index images, 560 image tokens* ↑ | 57.2 | 55.1 | −2.1 |
| Decision Index images, 280 image tokens* ↑ | 54.8 | 51.8 | −3.0 |
5. Memory and speed
iPhone 14 Pro Max (6 GB RAM), smaug-decisions-full-vis280, phone GPU, in the Cascade app:
| measured | |
|---|---|
| first load (one time, writes a cache) | 13-21 s |
| later loads | 0.48 s |
| text decision | ~0.6 s |
| audio decision | ~1.6 s |
| image decision (280 image tokens) | ~7.4 s |
| peak app memory, with Smaug-Audio-Decisions-0.7B loaded | 1.35 GB |
Storage: the file plus a cache of about 2.4 GB written on first load, so plan for about 6 GB.
6. Limitations
- Per request: 2,048 tokens including the prompt, one image, up to 20 options.
- Decisions only: the files answer typed questions; for text generation use full Smaug-Decisions.
- Thresholds: set decision thresholds on your own data for high-stakes uses.
7. Usage
Reading the decision. Prefill the prompt rendered by smaug_decide.render in abacusai/Smaug-Decisions
(no chat template re-applied) and take the logits of the option letters at the last position. Apply the softcap 30
(30 * tanh(z / 30)), then softmax over the options (one answer) or sigmoid(z - 8.0106) per option (select-all). Stock
LiteRT-LM's RunTextScoring, renormalized over the options, gives the one-answer probabilities; select-all needs the absolute logits,
which the runtime in the app repository exposes.
Cache. Put the cache written on first load in a directory the OS does not purge, and attach the vision encoder only when an image arrives.
Apps. The iOS (Swift) and Android (Kotlin) Cascade apps, including the device policy that picks the file, are at github.com/abacusai/Smaug-Decisions-Mobile.
8. License
Apache 2.0, the same license as the base model, Gemma 4 E4B-it.
9. Citation
@misc{abacusai2026smaugdecisions,
title = {Smaug-Decisions: calibrated typed decisions over audio, images and text},
author = {Abacus.AI},
year = {2026},
url = {https://huggingface.co/abacusai/Smaug-Decisions}
}
10. Contact Us
If you have any questions, please reach out at Abacus.AI.