Small decoder-only transformers trained from scratch on nothing but records of agents acting: tool definitions, tool calls, tool results and the text between them. They do not chat and were never asked to. The only thing measured is whether the model emits the right action.
Paper, scorer, training scripts and raw logs: https://github.com/thiagopatzdorf/action-is-all-you-need
Thiago Patzdorf and James — Genesis Innovation. Working paper, 12 November 2026.
Held-out trajectories, first tool call, greedy decoding, exact string match, no judge model. Rows are the 314 examples that offer two or more tools (chance = 31.5%).
| Model | Folder | Layers | Width | Parameters | Training tokens | Val. loss (bits/byte) | Right tool | Right tool and all arguments |
|---|---|---|---|---|---|---|---|---|
| A-126 | a-126/ |
8 | 512 | 125.8 M | 0.50 B | 0.125 | 88.2% | 55.4% |
| A-286 | a-286/ |
12 | 768 | 286.3 M | 1.32 B | 0.099 | 90.8% | 63.1% |
| A-537 | a-537/ |
16 | 1,024 | 536.9 M | 2.82 B | 0.079 | 94.0% | 68.5% |
| A-897 | a-897/ |
20 | 1,280 | 896.5 M | 5.22 B | 0.065 | 95.2% | 72.0% |
Total GPU cost: about US$ 25 for the first three runs and about US$ 61 for A-897, on one rented RTX PRO 6000.
A-897 is not a clean fourth point on the curve. It was trained on a larger corpus (the original data plus 138,723 resolved code-agent trajectories from Open-SWE-Traces and SWE-rebench), so size and data changed together. On a multi-step test with gold history it writes the whole call exactly in 61.5% of 579 calls (A-537: 56.7%); 65% of the calls in those trajectories do not fit in the 2,048-token context and are not scored.
On those same 579 calls, against a production general-purpose model given the same input (tool name and arguments compared as objects): A-897 alone 61.7% whole call right at 0.20 s per call; the general model alone 59.8% at 1.03 s; a pair in which the general model answers only when A-897 is unsure (27.6% of the calls) 64.8%. These differences are within noise and the format is A-897's home format: read it as "comparable, faster, complementary", not "better".
a-126/, a-286/, a-537/, a-897/: model_*.pt (PyTorch state dict) and meta_*.json (configuration), as written by
nanochat.tokenizer/: tokenizer.pkl and token_bytes.pt. tokenizer.pkl is a Python pickle, the format nanochat
writes. Load it only if you trust this repository.These are checkpoints of the open nanochat codebase (commit in
nanochat-commit.txt in the GitHub repository). Place a model folder under $NANOCHAT_BASE_DIR/base_checkpoints/
and the tokenizer under $NANOCHAT_BASE_DIR/tokenizer/, then load it as a nanochat base model. The prompt is the list
of available tools followed by the message sequence, in the format produced by the data-preparation scripts in the
GitHub repository; regua_b.py there is the scorer that produced the table above.
Research artifacts. Next-token predictors over agent trajectories: not instruction-tuned, 2,048-token context, not evaluated for safety or for production use. A model that emits tool calls should never be connected to tools that can change real systems without a human or a policy layer in between.
Weights and code: Apache-2.0. Paper text: CC BY 4.0.
@misc{patzdorf2026action,
title = {Action Is All You Need},
author = {Patzdorf, Thiago and James},
year = {2026},
note = {Genesis Innovation working paper},
url = {https://github.com/thiagopatzdorf/action-is-all-you-need}
}
a-897/model_004980.pt 2.7 GBa-537/model_005376.pt 1.5 GBa-286/model_002520.pt 793 MBa-126/model_001920.pt 336 MBtokenizer/token_bytes.pt 132.6 kB