Multilingual Punctuation + Truecasing (base, 307M): 88 languages

Restores punctuation and capital letters in lowercase text in one pass, for example the output of a speech-recognition (ASR) system, subtitles or chat messages. For each word it predicts the mark that follows it (. , ? : - or nothing) and its casing (lower, Capitalized or UPPER). Trained on 88 languages. Built on mmBERT-base, Apache-2.0, ONNX files for CPU and the browser (transformers.js) included. Try it in the browser. A smaller, faster version is available as multilingual-punctuation-truecase-small.

Actual output of this model (from the release check):

in : my name is anna i live in berlin and work for the united nations in new york
out: My name is Anna, I live in Berlin and work for the United Nations in New York.
in : ich heiße thomas und wohne in münchen was machst du am wochenende
out: Ich heiße Thomas und wohne in München. Was machst du am Wochenende?

• A standard token-classification model with 18 labels <mark>|<case> (0|lower, .|Cap, ,|UPPER, ...), so it runs with the transformers pipeline and transformers.js. The included truecase.py turns the labels into text. It also writes ，。？： in Chinese and Japanese (no word segmentation needed), । in Hindi and Bengali, ، ؟ in Arabic script, ; for questions in Greek and ። ፣ in Amharic.
• Input is lowercased first: the model was trained on lowercase text and does not use casing you already have. If your text is already cased and only lacks punctuation, use punctuation-restoration-base instead.
• onnx/model_quantized.onnx (int8 embeddings, 641 MB) predicts the same label as fp32 for 99.92% of tokens on lowercased held-out text.

Usage

With the included helper (any language; long texts are processed in overlapping chunks):

from huggingface_hub import hf_hub_download
import importlib.util
spec = importlib.util.spec_from_file_location("truecase", hf_hub_download("Horizon-Labs/multilingual-punctuation-truecase-base", "truecase.py"))
truecase = importlib.util.module_from_spec(spec); spec.loader.exec_module(truecase)

t = truecase.Truecaser("Horizon-Labs/multilingual-punctuation-truecase-base")
print(t("my name is anna i live in berlin and work for the united nations in new york"))
print(t("今天天气很好我们去公园散步吧你觉得怎么样"))

Plain transformers: the label of each word is the label of its last token. Lowercase the text first.

from transformers import pipeline
tagger = pipeline("token-classification", model="Horizon-Labs/multilingual-punctuation-truecase-base")
tagger("the nasa mission launched on monday from florida")

transformers.js:

import { pipeline } from "@huggingface/transformers";
const tagger = await pipeline("token-classification", "Horizon-Labs/multilingual-punctuation-truecase-base", { dtype: "q8" });
console.log(await tagger("the nasa mission launched on monday from florida"));

Evaluation

Every system gets the same input: a paragraph lowercased, with . , ? : ; ! - removed. The test sets were used only for evaluation:

• Europarl: European Parliament proceedings, 5 consecutive sentences, 200 paragraphs in each of 12 languages.
• TED talks (TED2020 via OPUS): transcripts and their translations, 5 consecutive sentences, 200 paragraphs in each of 25 languages. Spoken style.
• FLORES-200 devtest: professionally translated news, travel and wiki text, one paragraph per article. Columns: our 88 training languages, and all 199 FLORES languages (the extra 111 were never trained on).

Metrics are averaged over languages:

• Casing F1: over the words that should be Capitalized or UPPER. A word counts only with the exact class, and capitals predicted on words that should be lower case count as false positives. Languages are included when their test text contains capitals (for Chinese, Japanese, Arabic and other scripts without case, only the few Latin-script words count).
• Casing accuracy: all words, including the lower-case ones.
• Punctuation F1: as on the punctuation-restoration cards. Micro-F1 over the five marks, and a mark counts only at the right position.

Casing F1:

Punctuation F1 (lowercased input):

Casing accuracy:

• + rule casing is our punctuation-only model punctuation-restoration-base, followed by capitalizing the first word and every word after . or ? (and English "i"). This is what you get by adding capitals to the output of a punctuation-only model. Its punctuation F1 is that model's own lowercased-input score on these paragraphs.
• xlm-roberta_punctuation_fullstop_truecase (Apache-2.0, 47 languages) was run through the punctuators package on lowercased input, with its sentence splitting turned off, and its output was re-aligned to the input words. 2.3% of words could not be aligned and count as "no mark, lower case", which lowers its scores somewhat. The FLORES columns include languages it was not trained on.
• Two training seeds per size were evaluated before release. Mean casing F1 / punctuation F1 over Europarl, TED and FLORES-199: small .824 / .705, base .834 / .720.

Per language (FLORES, our 88 training languages)

Training

• Data: the training text of punctuation-restoration v1.1: 1,168,626 windows (119M words) of FineWeb-2 and FineWeb web text (ODC-BY) in 88 languages, plus 49,144 synthetic spoken-style transcripts in 69 languages, written by Qwen3.8-27B (Apache-2.0) for this purpose. Any transcript that shared a sentence with a test set was removed.
• Labels come from the original text. Punctuation is normalized as for punctuation-restoration. The casing classes are: Cap (first cased letter upper case), UPPER (two or more cased letters, all upper case) and lower (everything else, including words in scripts without case). The model always sees the text lowercased.
• Model: mmBERT-base token classification, 18 joint labels on each word's last token, 1 epoch, learning rate 3e-5 (the punctuation-restoration-base recipe). The checkpoint and the released seed were chosen on held-out web text by the mean of punctuation F1 and casing F1.
• Release criteria, fixed before training: punctuation F1 no more than .01 below punctuation-restoration-base on lowercased input; casing F1 at least .05 above the rule baseline on every test set and .10 above on average; and higher casing F1 than xlm-roberta_punctuation_fullstop_truecase on Europarl + TED. Both sizes passed.
• Code: code/ in this repository.

Limitations

• Only three casing classes. Mixed-case words come out wrong: iphone becomes Iphone, and mcdonald's becomes Mcdonald's. All-caps words are rare (0.3% of the words in the test text), so acronyms the model has not seen may stay lower case.
• Capitals on names depend on the model having seen the name or its context. This is weaker in low-resource languages (see the per-language table).
• The mark after a word and the casing of the next word are separate labels, so the model sometimes writes a comma followed by a capital ("Hello, How are you") where a full stop belongs or the next word should be lower case. On the test sets this happens for about 1 in 1,000 words or fewer, and most comma + capital predictions are right (a name after a comma). A decoding that re-checks these boundaries did not help on held-out text (eval/boundary_decoding.json), so truecase.py uses the plain predictions.
• Your casing is ignored: the input is lowercased first.
• Only . , ? : - are predicted. Exclamation marks, semicolons, quotes and brackets are not restored, and : and - are much less reliable than . , ?. Thai, Lao, Khmer, Burmese and Tibetan are not supported.
• Trained mostly on written web text, plus synthetic spoken-style transcripts. Disfluent speech (fillers, false starts, repetitions) is harder.