ATF-to-English (T5-base, H2): Sumerian, Akkadian and Hittite

Translates cuneiform transliterations (ATF-style, Latin script) into English. One model covers Sumerian, Akkadian (Old Akkadian through Neo-Babylonian, now including Neo-Assyrian letters and royal inscriptions) and Hittite. It is a 223M-parameter T5-base fine-tune and the model behind the English on The Unread Atlas.

This is stage H2, a continuation of stage H (commit bc8c849) with added Neo-Assyrian training data. The T5-small model montycrypto/atf-to-english stays as it is. Code and earlier training scripts: github.com/ixmon/atf-to-english. Site code: github.com/ixmon/unread.

What's new in H2

• Neo-Assyrian letters and royal inscriptions. About 21k new ORACC pairs from 3,225 texts. On 320 held-out tablets, letters go from 44.3 to 50.5 chrF (BLEU 22.0 → 34.0) and royal inscriptions from 44.7 to 51.9 chrF (BLEU 15.0 → 23.1), now clearly ahead of the T5-small model (44.8 and 47.9 chrF).
• Unicode ATF input works directly. Most training ATF is CDLI-style ASCII (sz, s,, no damage brackets), and stage H lost several points when fed Unicode ATF (š, ₂, ⸢⸣, [ ]) as published by ORACC. H2 was trained on both forms and scores within 0.7–1.3 chrF of its best preprocessing on raw Unicode input, so no conversion step is needed.
• Hittite improves from 20.4 to 23.4 BLEU (40.3 → 43.7 chrF) on the same held-out texts.
• Assyrian royal names. On 1,906 held-out lines naming Neo-Assyrian kings, the king's name comes out correctly 64.6% of the time (details below).
• Mixed Sumerian/Akkadian scores are unchanged from H (line chrF 49.09 vs 49.13).

Usage

The codec is part of the model's input format: apply encode() before tokenizing and decode() after generating. Without it, any line containing š, ḫ, {d} etc. is degraded. The prompt prefix translate Akkadian to English: is used for every language, including Sumerian and Hittite. Unicode ATF (š, ₂, ⸢⸣, [ ]) and CDLI-style ASCII ATF are both accepted.

import re, unicodedata
from transformers import AutoTokenizer, T5ForConditionalGeneration

MODEL = "montycrypto/atf-to-english-t5base"

# --- input/output codec (required) ---------------------------------------
# The stock T5 vocabulary has no š, ṣ, ṭ, ḫ, ā, {, } ... so every character it
# cannot round-trip is escaped to ASCII before tokenizing, and the output is
# unescaped afterwards. The model was trained on escaped text on both sides.
MARKS = {"\u030c": "cv", "\u0304": "mc", "\u0302": "cf", "\u0301": "ac", "\u0300": "gr",
         "\u0323": "db", "\u032e": "bb", "\u0308": "um", "\u0303": "td", "\u0307": "da",
         "\u0327": "ce", "\u0331": "lb", "\u0306": "br"}
SPECIAL = {"{": "ob", "}": "cb", "ʾ": "al", "ʿ": "ay"}
RMARKS = {v: k for k, v in {**MARKS, **SPECIAL}.items()}
SUB = str.maketrans("₀₁₂₃₄₅₆₇₈₉ₓ", "0123456789x")
SAFE = set(chr(i) for i in range(32, 127)) - set("{}^~`<\\&")

def _esc(ch):
    if ch in SAFE: return ch
    if ch == "&": return "&&"
    if ch in SPECIAL: return "&" + SPECIAL[ch]
    if ch in "\n\t": return " "
    d = unicodedata.normalize("NFD", ch)
    if len(d) > 1 and d[0] in SAFE and all(m in MARKS for m in d[1:]):
        return d[0] + "".join("&" + MARKS[m] for m in d[1:])
    return "&u%04x" % ord(ch) if ord(ch) < 0x10000 else "&U%06x" % ord(ch)

def encode(atf):
    s = unicodedata.normalize("NFC", atf).translate(SUB)   # subscript digits -> plain digits
    return "".join(_esc(c) for c in s)

_RX = re.compile(r"&(&|u[0-9a-f]{4}|U[0-9a-f]{6}|[a-z]{2})")
def decode(s):
    def r(m):
        g = m.group(1)
        if g == "&": return "&"
        if len(g) > 2: return chr(int(g[1:], 16))
        return RMARKS.get(g, m.group(0))
    return unicodedata.normalize("NFC", _RX.sub(r, s))
# --------------------------------------------------------------------------

tok = AutoTokenizer.from_pretrained(MODEL)
model = T5ForConditionalGeneration.from_pretrained(MODEL).eval()

def translate(lines):
    batch = ["translate Akkadian to English: " + encode(l) for l in lines]   # same prefix for every language
    enc = tok(batch, return_tensors="pt", padding=True, truncation=True, max_length=512)
    out = model.generate(**enc, max_new_tokens=256, num_beams=4, early_stopping=True,
                         no_repeat_ngram_size=3, repetition_penalty=1.2)
    return [decode(t) for t in tok.batch_decode(out, skip_special_tokens=True)]

examples = [
    "1(diš) udu niga {d}inanna",                                # Sumerian (Ur III offering formula)
    "a-na {d}UTU-ha-zi-ir qi₂-bi₂-ma um-ma {d}AMAR.UTU-na-ṣir-ma",  # Old Babylonian letter opening
    "{d}PA u {d}AMAR.UTU a-na LUGAL EN-ia lik-ru-bu",            # Neo-Assyrian letter greeting
    "É.GAL {m}aš-šur-ŠEŠ-SUM-na LUGAL KUR aš-šur",               # Neo-Assyrian royal inscription
    "LUGAL-uš MUNUS.LUGAL-ya TUŠ-aš e-ku-zi",                    # Hittite (festival formula)
]
for src, en in zip(examples, translate(examples)):
    print(f"{src}\n  -> {en}")

Output:

1(diš) udu niga {d}inanna
  -> 1 sheep, grain-fed, for Inanna
a-na {d}UTU-ha-zi-ir qi₂-bi₂-ma um-ma {d}AMAR.UTU-na-ṣir-ma
  -> Speak to Šamaš-hazir, thus says Marduk-nāṣir.
{d}PA u {d}AMAR.UTU a-na LUGAL EN-ia lik-ru-bu
  -> May Nabû and Marduk bless the king, my lord!
É.GAL {m}aš-šur-ŠEŠ-SUM-na LUGAL KUR aš-šur
  -> Palace of Esarhaddon, king of Assyria.
LUGAL-uš MUNUS.LUGAL-ya TUŠ-aš e-ku-zi
  -> The king and the queen drink while seated.

Input preprocessing, exactly

1. Unicode NFC normalisation.
2. Subscript digits and ₓ become plain ASCII (be-li₂ → be-li2, du₃ → du3). Write sign indices either way.
3. Every character outside printable ASCII, plus {, }, ^, ~, `, <, \ and &, is escaped:
   • a letter with combining marks becomes the base letter plus & and a 2-letter code per mark (š → s&cv, ḫ → h&bb, ṣ → s&db, ā → a&mc, é → e&ac);
   • { → &ob, } → &cb, ʾ → &al, ʿ → &ay, a literal & → &&;
   • anything else → &uXXXX (hex code point); newlines and tabs become spaces.
4. Prefix translate Akkadian to English: ; truncate at 512 tokens.
5. Outputs are passed through decode(), which inverts step 3 exactly.

Other ATF conventions are kept as in the sources: {d} determinatives, logograms in capitals (Hittite, CDLI) or _..._ (ORACC), [...] for breaks, ⸢⸣ for damaged signs, x for unreadable signs, 1(diš) style numbers.

Decoding settings

num_beams=4, early_stopping=True, no_repeat_ngram_size=3, repetition_penalty=1.2, max_new_tokens=256 (512 for whole tablets). These are also the defaults in generation_config.json, and they are the settings used for every score below. No custom logits processors are needed: on the regression set this decoding produced no repetition loops.

Training

Four stages, all with the codec applied to both source and target, bf16, AdamW, cosine schedule, weight decay 0.01, effective batch 32 (4 × 8 gradient accumulation), gradient checkpointing, max length 512, on one NVIDIA GB10 (DGX Spark).

The Neo-Assyrian data. 20,952 pairs (16,306 translation units, 3,483 chunks, 1,163 whole texts) from 3,225 ORACC texts: mainly letters from the State Archives of Assyria online (SAAo) and ATAE, and royal inscriptions from RINAP, RIAo and RIBo, including 91 RINAP 5 Ashurbanipal texts. Source lines were aligned to ORACC's translation units by line label and checked against the edition's own ATF; poorly aligned units were dropped. In the H2 mix these were given in CDLI-style ASCII form, royal units twice, plus 8,000 units repeated in their original Unicode form, about 25% of the mix. Held-out tablets, texts with near-parallel exemplars, and all earlier evaluation texts were removed from both the new data and the replay.

The Sumerian/Akkadian pool from earlier stages is 239,298 cleaned line pairs (misaligned ORACC pairs and duplicates removed), of which 234,806 were usable after the held-out sets were removed.

Data sources and credit

How the Hittite pairs were made. The Yavasan & Gordin dataset gives word-level German glosses for 7,099 HPM texts. Each glossed line was aligned to the full TLHdig Beta 0.3 transliteration of the same line, and the German glosses were rendered as fluent English line translations with a local Qwen LLM. Lines where German leaked through, or where the English came back empty, were dropped. 270 whole texts (2,823 lines) were held out and never trained on.

This repository contains only the model weights, tokenizer and configuration. No training data is redistributed.

Evaluation

sacrebleu corpus BLEU / chrF, decoding as above.

Neo-Assyrian. 320 ORACC tablets held out from all training: 200 letters (199 SAAo, 1 ATAE) and 120 royal inscriptions (82 RINAP, 26 RIAo, 6 RIBo, 6 other). Each tablet is translated line by line from its Unicode ATF exactly as ORACC publishes it and scored against the edition's English. The T5-small model is given CDLI-style ASCII, the input it was trained on.

With CDLI-style ASCII input and damage marks removed, H2 scores 51.22 / 34.11 on letters and 53.20 / 24.04 on royal inscriptions, so raw Unicode input costs little. Letter formulas recur from text to text, so about 22% of held-out letter lines (14% of royal lines) of five or more signs also occur in other, training texts.

Royal names. 1,906 held-out lines that spell a Neo-Assyrian king or Mullissu (1,997 names in all), never trained on. A name counts as correct when the output contains an accepted English rendering.

The Unread Atlas adds a rule-based name correction for Esarhaddon, Ashurbanipal and Sargon on top of the model, which raises the total to 85.7%.

Sumerian and Akkadian. 1,500 held-out lines (ORACC 399, CDLI 727, CuneiformTranslators 326, synthetic 48), disjoint from training by normalised transliteration, plus 300 ORACC tablets (6–30 lines each) held out entirely.

The T5-small model still scores higher on single-line BLEU, mostly on short formulaic lines; the T5-base stages are much stronger on running text and whole tablets, and they keep names with diacritics intact.

Hittite. 270 held-out texts, 2,823 lines. References are English renderings of the German HPM translations (see above), so they are themselves translated once.

Confidence scores

A small calibrator predicts each line's chrF from the model's own signals: mean token log-probability, agreement between the beam output and six sampled translations, and line length and damage. On the 1,452 human-translated lines of the mixed test set (cross-validated) its rank correlation with true line chrF is 0.71 (Spearman), and it flags lines below 25 chrF with AUROC 0.85. Lines it rates most confident average 60 chrF; the least confident tier averages 16. It is much weaker on Neo-Assyrian passages scored against ORACC's paragraph-level translations (Spearman about 0.2–0.3). The Unread Atlas is moving its line shading from a rule-based score to this one. The calibrator is not part of this repository.

Known weak spots

• Ashurbanipal, Esarhaddon and Sargon are often missed or garbled in Assyrian spellings, and Sennacherib is right only about two times in three.
• On Neo-Assyrian input the model sometimes adds ORACC-style restoration brackets ([king]) where the source is not broken.
• Hittite vocabulary outside offering and festival texts is thin: unuwanzi "they adorn" can come out as "they bring", and some professions are confused ({LÚ}AZU "ritual expert" as "cupbearer").
• Ugaritic and other alphabetic cuneiform are not supported.
• Rare words, broken passages and personal names are the most common sources of error.

Citation

@misc{montycrypto-atf-to-english-t5base,
  title     = {ATF-to-English (T5-base): Sumerian, Akkadian and Hittite transliteration to English},
  author    = {montycrypto},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/montycrypto/atf-to-english-t5base}
}

Please also cite the data: ORACC, CDLI, CuneiformTranslators, Yavasan & Gordin (2024) and TLHdig (Müller et al. 2026).