ATF-to-English Translation (T5-large)

Current version: B2c (2026-10-27), T5-large (~738M parameters), checkpoint t5large-B2c/checkpoint-22000.

Earlier versions stay available as revisions:

• v4e (T5-small, 2026-10-11): tag v4e-2026-10-11 (from_pretrained("montycrypto/atf-to-english", revision="v4e-2026-10-11"))
• v2 (T5-small, original): tag v2-2026-10-07

Translates cuneiform transliterations (ATF format) into English. Handles Akkadian (Old Assyrian through Neo-Babylonian), Sumerian, Hittite and mixed-language texts spanning ~3,000 years, from Presargonic and Ur III administrative texts to Neo-Assyrian letters, royal inscriptions, literature and lexical lists.

Usage

from transformers import AutoTokenizer, T5ForConditionalGeneration

model = T5ForConditionalGeneration.from_pretrained("montycrypto/atf-to-english")
tokenizer = AutoTokenizer.from_pretrained("montycrypto/atf-to-english")

text = "translate Akkadian to English: um-ma a-szur-i-di-ma a-na pu-szu-ke-en6 qi2-bi4-ma"
inputs = tokenizer(text, return_tensors="pt", max_length=512, truncation=True)
outputs = model.generate(**inputs)   # decoding defaults are in generation_config.json
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
# → "Thus (says) Aššur-idi: speak to Pušu-ken:"

Input format

• One line per call, prompt prefix translate Akkadian to English: (used for all languages).
• CDLI-style ASCII ATF works best: sz for š, s, for ṣ, t, for ṭ, j for ĝ, sign indices as digits (u2, bi2), determinatives in braces ({d}, {ki}, {lu2}, {m}), broken text in [ ]. Unicode ATF (š, ṣ, ú, subscript digits) is understood but was normalised to ASCII during training.
• Optional context (how the model was mostly trained): append neighbouring lines and English gloss hints, <line> =P= <previous line> =N= <next line> =G= <gloss; gloss>. The bare line alone also works well (about 1 chrF lower on the mixed set).
• The tokenizer extends T5's vocabulary with ATF pieces ({d}, {ki}, <<, >> …) and diacritic letters (š ā ī ū ṣ ṭ ḫ …), so names come out with correct diacritics. Rare characters with no piece may appear as &uXXXX escapes (Unicode code point).

Recommended decoding

generation_config.json sets the decoding used for the evaluation below: num_beams=4, early_stopping=True, max_new_tokens=256, repetition_penalty=1.2, no_repeat_ngram_size=3.

Examples (B2c)

Training

B2c (2026-10-27)

• Base: google-t5/t5-large with the extended ATF tokenizer, trained in stages (large → B1 → B2c); B2c continues from B1 (best checkpoint, eval_loss 0.954).
• Data: 454k training examples drawn from a cleaned pool of ATF–English line pairs (ORACC, CDLI, eBL-derived pairs, CuneiformTranslators, Hittite) plus 18k new pairs from CC0 ORACC projects (ribo, hbtin, cams/gkab, etcsri, blms, obmc, adsd, ccpo, rimanum) and CDLI Ur III English translations. Cleaning removed misaligned, wrong-language and placeholder targets. All evaluation sets are excluded from training by key.
• Mixture: 45% lines with context (neighbours + glosses), 40% bare lines, 12% multi-line chunks, 3% whole tablets; rare-sign lines and plain Old Babylonian / Ur III lines upsampled 2×.
• Schedule: LR 3e-5 constant with warmup, effective batch 32, bf16, early stopping on eval loss (best step 22,000, eval_loss 0.901).
• Hardware: NVIDIA GB10 (DGX Spark).

v4e (2026-10-11) and v2 (2026-06)

T5-small (60M). v4e: 239k cleaned pairs, initialized from v2. v2: 131k curated pairs from eBL, CDLI, ORACC and CuneiformTranslators with synthetic rare-vocabulary augmentation. See the tagged revisions for their cards.

Evaluation

Cleaned held-out gate sets, disjoint from training. Scores with numbers normalised; the headline buckets leave out placeholder references, lexical-list lines and very short lines (≤4 signs), reported separately. Δ with 95% paired bootstrap CI (1,000 resamples). COMET = wmt22-comet-da. Compared with B1 (the previous T5-large stage) and H2 (the T5-base production model).

By rare-sign quartile (mixed set, chrF vs H2): Q1 +5.90, Q2 +5.16, Q3 +4.22, Q4 +2.41, all significant. Hittite references are model-made, so the Hittite row measures agreement with those references.

Known weak spots

• Brackets for damaged text: B2c writes fewer [ ] than the references, which costs some BLEU on Neo-Assyrian letters with bare-line input.
• Lexical lists: some glosses are off (e.g. šinig "thornbush" instead of tamarisk, kisega "banquet" instead of funerary offering).
• Old Assyrian: the pronoun a-ta ("you") is sometimes confused with "why" in letters.
• Line-level model: very long multi-line passages may lose coherence; period-specific sign readings can still be mixed up without context.

Citation

@misc{montycrypto-atf-to-english,
  title={ATF-to-English Translation Model},
  author={montycrypto},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/montycrypto/atf-to-english}
}