masterofaudio2077/byt5-small-arabizi-word

byt5-small-arabizi-word

ByT5-small fine-tuned on arbml/Arabizi_Transliteration: 19,514 Gulf- and Levantine-style word pairs. One model, both directions. Put a prefix in front of the word and it converts Arabizi to Arabic script or Arabic script to Arabizi. Single words only.

Examples

Arabizi → Arabic, top-1 output of this model:

Usage

import torch
from transformers import AutoTokenizer, T5ForConditionalGeneration

MODEL_ID = "masterofaudio2077/byt5-small-arabizi-word"
tok = AutoTokenizer.from_pretrained(MODEL_ID)
model = T5ForConditionalGeneration.from_pretrained(MODEL_ID).eval()

PREFIX = {"arabizi_to_arabic": "to arabic: ", "arabic_to_arabizi": "to arabizi: "}


def convert(word, direction, n=1):
    enc = tok(PREFIX[direction] + word, return_tensors="pt")
    with torch.no_grad():
        out = model.generate(**enc, num_return_sequences=n)   # beam search, 5 beams, at most 40 bytes out
    return [t.strip() for t in tok.batch_decode(out, skip_special_tokens=True)]


print(convert("7abibi", "arabizi_to_arabic"))
print(convert("حبيبي", "arabic_to_arabizi", n=3))

The generation settings (5 beams, 40 new tokens) are saved in the repo, so a plain model.generate reproduces the numbers below. Arabic → Arabizi has several valid spellings per word; use n=3 or n=5 to get the top candidates.

Prefixes

Results

1,000 held-out rows: 988 unique Arabizi words and 689 unique Arabic words. A prediction counts as correct if it equals any spelling the dataset knows for that word.

Round trip (Arabic → Arabizi → Arabic, lenient): 73.9%.

By source length (lenient exact match for → Arabic, exact match for → Arabizi):

Most frequent errors: Arabizi → Arabic mostly adds or drops ا and ي (71, 67, 42, 27 cases), plus ط/ت and س/ص confusions. Arabic → Arabizi mostly drops or swaps the vowels a, e, i, o.

Training

• Full fine-tune of google/byt5-small (300M, byte-level), 10,000 steps at batch 128 on a Kaggle GPU (6.9 h), Adafactor (lr 1e-3, 200 warm-up steps then linear decay), label smoothing 0.1, fp32. This is step 2000, the best checkpoint on validation.
• 17,514 training pairs, 1,000 validation, 1,000 test. Rows are split by connected components of (Arabic word, Arabizi string), so no word appears in two splits. Every pair is used in both directions.
• Cleaning: removed 3 Excel-date rows, 3 digit-only sources, 1 row with no Latin letter and 1,978 duplicate pairs. Arabizi is lower-cased and NFKC-normalised; Arabic has diacritics and tatweel removed.
• Weights are published in fp32.

Limitations

• Single words only. Training sources were at most 13 Arabizi and 12 Arabic characters.
• Follows the conventions of its data: Gulf and Levantine chat (8 = ق, 9' = ض, $ = ش). The dataset card gives no provenance, so this comes from inspecting the data. Other dialects' conventions may be converted poorly.
• The dataset declares no licence. Check that your use is allowed.
• The references spell the same word with ة/ه, ا/أ/إ and ي/ى inconsistently, and Arabic → Arabizi has many valid spellings, so exact match understates quality. Read it together with the top-5 figure.
• The test split is our own group split, not a community benchmark.

Acknowledgements

ByT5 by Google (paper) and the arbml/Arabizi_Transliteration dataset.