Amos 1 small
A 2B decision model. Give it a case (text, JSON or a photo) and a schema of typed questions; it answers every question in one forward pass, each with a probability for every answer you allow. It writes no text, so there is nothing to parse and no malformed output. Built for volume. It reads text, JSON and photos, quotes exact phrases and points at things in photos, all in the same pass as its answers.
Part of the Amos 1 family: large · medium · small · mini. Playground · API docs · Model page
Results
Typed decisions, test split: 2,000 decisions from four workflows it never trained on, zero-shot.
| Accuracy | KL ↓ | Brier ↓ | |
|---|---|---|---|
| Amos 1 small | 69.5% | 0.181 | 0.098 |
| Microsoft-Decision-1 (closed weights) | 79.5% | 0.318 | |
| Decider 1 (its makers' number) | 76.8% | 0.096 | |
| Perplexity Decider v1.1 | 73.9% | 0.466 | |
| Clef (Cloudflare) | 72.5% | 0.191 |
Rivals run by us on the same 2,000 decisions, except Decider 1, whose number is its makers' own.
With each question's priors fitted on the benchmark's own train split and applied at answer time (the weights do not change), as Decider 1 also reports: 74.4% (Decider 1: 80.1%).
Decision Index 0.3, public part, with the board's own kit: 31.92. The board's ranking adds private tests that only its maintainers run.
| Other benchmarks, as served | |
|---|---|
| JevBench public easy / standard / hard | 100.0 / 93.1 / 53.2 |
| Ten public benchmarks (RAGTruth, JudgeBench, Belebele, WinoGrande, ContractNLI, TruthfulQA, Circa, Financial PhraseBank, TabFact, BBH) | 72.1% |
| AG News / DAIR Emotion / Banking77 (77 labels) | 86.5 / 58.5 / 76.0 |
| Invoice totals quoted exactly | 9 / 10 |
| Box centred on the named object, RefCOCO | 94.2% |
| Box covering at least half the object, RefCOCO | 77.2% |
| Box centred on the named object, COCO | 97.5% |
| Server time, median, three-question request | 48 ms |
Use
Hosted, no key: POST https://api.amos.scira.ai/v1/systemone with "model": "amos-1-small".
Or run it yourself. The serving package is in serve/ (Python 3.12, PyTorch 2.14; versions pinned to the ones these
numbers were measured with):
hf download mukaddamzaid/amos-1-small --local-dir amos-1-small
pip install ./amos-1-small/serve
python -m amos.serve amos-1-small --device cuda --port 8000 --max-len 262144
curl http://127.0.0.1:8000/v1/systemone -H 'Content-Type: application/json' -d '{
"state": "Invoice INV-2231 for £306.00 was charged to my card twice. Please refund one.",
"questions": {
"refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"},
"team": {"type": "choice", "instructions": "Which team should handle this?",
"criteria": {"billing": null, "technical": null, "sales": null}}
}
}'
Question types: yes/no (noul), pick one (choice, up to 255 options), rate (score), pick any (multi), quote
(span) and point (region, with a photo in images). It runs on an 8 GB GPU and reads up to
262,144 tokens per request; cases near that length need about 12 GB.
| File | What |
|---|---|
model/ (weights, tokenizer, image processor, heads.pt) |
The model |
calibration.json |
Quote decoding settings the server reads |
serve/ |
The serving package (amos.serve) |
typed_decisions.json |
The typed-decisions test scores above |
Limits
- Seven points behind medium on typed decisions.
- Weaker on questions that need broad knowledge.
- Ratings are its least reliable answers.
- English only.