Loading…
Loading…
acpi-ai's 106.4M-parameter on-device language model with a 9.1 GB statistical
memory tier. It speaks English and Traditional Chinese, and fits a 128 MB weight budget built for edge hardware (32.7 B tokens total, from a from-scratch tokenizer up).
The "n9g" in the name is the second half of the system: a 5-gram trie (ngram_5gram.bin) that
lives on SSD, is consulted per reply by fusion rescoring.
| Property | Specification |
|---|---|
| Type | Causal LM, dense decoder-only; ships in Llama-compatible format |
| Total parameters | 106,390,272 (tied-emb core; 130,966,272 with untied LM head after SFT) |
| Hidden size / layers | 768 × 16 |
| Attention | GQA, 12 query heads / 4 KV heads, RoPE θ=500,000 (half-split), RMSNorm ε=1e-6 |
| FFN | SwiGLU, intermediate 1,536 |
| Tokenizer | sentencepiece BPE, 32,000 tokens (EN + zh-TW) |
| Embeddings | tied input/output (shipped head duplicate is numerically identical, maxdiff 0.0) |
| Context | 2,048 tokens |
| Precision shipped | bfloat16 safetensors — 212.8 MB (≈53 MB at 4-bit) |
| Memory tier (n9g) | pruned 5-gram KenLM trie, quantized q8, 9,134,143,128 bytes, mmap'd from SSD |
Harnesses were committed before the final evaluation run; every number below is from logged runs:
| Gate | Result | Notes |
|---|---|---|
| Fresh-document val PPL | 114.65 | doc-held-out 232M-token set; pre-CT checkpoint scored 4,338 (37.8× generalization gain) |
| Conversational junk rate (mouth / system / 9B brain) | 32.1% / 28.6% / 17.9% | 28-probe adversarial suite, 3-tier ablation |
| Punt-router separation | 4 orders of magnitude | fluent 15-50 vs junk ≥10⁶; reference line 250 |
| Fusion vs solo (same weights, switch flipped) | junk 39.3% → 32.1% | 5 rescues, 3 regressions, net −2; fusion costs ~6× decode latency |
Good: answering as ant-128m-n9g from acpi-ai (EN + zh-TW); fluent short bilingual chat; flagging its own junk replies with a usable confidence number; running anywhere Llama runs, including CPU-only and RAM-poor devices (the n-gram degrades to a few hundred MB working set); shipping a rebuildable statistics tier nobody else has.
Not: arithmetic and factual knowledge (128M-class ceiling — MMLU lands ~25-35; pair with a brain tier); long coherent stories; open-ended chat under greedy decoding, where the identity line can act as an attractor and loop; languages other than English and Traditional Chinese.
ant-128m-n9g is a probabilistic language model trained on statistical patterns from public web text (scraped ≤ 2025, deduplicated, doc-held-out validation throughout).
Setup
pip install .
(torch, transformers, kenlm-ex — all prebuilt wheels, no compilers, no binaries shipped in this folder.)
Run
python example.py # built-in: "hello how are you"
python example.py "your prompt here" # anything else
Works on CPU; uses CUDA automatically if present.
Files
| file | what |
|---|---|
| model.safetensors, config.json, tokenizer files | the 128M model (212.8 MB bf16, ≈53 MB at 4-bit) |
| ngram_5gram.bin | the n9g memory tier (9.1 GB, mmap'd, read-only) |
| setup.py | dependency install |
| example.py | minimal fused-decode inference |