MarianMT Syriac Vocalization Model

A fine-tuned MarianMT model for automatic Syriac vocalization, converting consonantal (unvocalized) Syriac text to fully vocalized text with diacritical marks.

Model Description

This model is fine-tuned from Helsinki-NLP/opus-mt-tc-bible-big-sem-en to perform Syriac vocalization—the task of adding diacritical marks (vowels) to consonantal Syriac text. The model is trained in a single direction: consonantal → vocalized.

Key Features

• Single-direction model: Converts consonantal Syriac (>>syr_cons<<) to vocalized Syriac (>>syr_voc<<)
• Leverages pretrained Syriac tokenization: Built on a model that already includes >>syr<< tokenization
• High performance: Achieves 67.60 BLEU, 90.41 chrF, and 60.23% character accuracy on test set
• Biblical text optimized: Trained on biblical Syriac texts for accurate vocalization

Model Details

Model Information

• Architecture: MarianMT (Transformer-based sequence-to-sequence)
• Base Model: Helsinki-NLP/opus-mt-tc-bible-big-sem-en
• Parameters: 240,944,128 (~241M)
• Vocabulary Size: 61,025 tokens
• Language Tags:
  • Source: >>syr_cons<< (consonantal Syriac)
  • Target: >>syr_voc<< (vocalized Syriac)

Training Data

• Training Examples: 26,997
• Validation Examples: 3,104
• Test Examples: 931
• Total: 31,032 sentence pairs
• Source: Biblical Syriac texts (consonantal and vocalized pairs)

Training Configuration

• Batch Size: 8
• Effective Batch Size: 32 (with gradient accumulation)
• Learning Rate: 1e-5
• Max Input/Target Length: 512 tokens
• Training Steps: 39,000
• Epochs: 46.2
• Optimizer: AdamW with cosine learning rate schedule
• Precision: bfloat16
• Early Stopping: 5 evaluation calls without improvement
• Best Checkpoint: Step 34,000

Performance

Best Validation Metrics (Step 34,000)

• BLEU: 67.13
• chrF: 90.16
• Character Accuracy: 59.75%
• Validation Loss: 0.0563

Final Test Metrics

• BLEU: 67.60
• chrF: 90.41
• Character Accuracy: 60.23%
• Test Loss: 0.0520

Usage

Direct Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("johnlockejrr/marianmt_syr_voc")
model = AutoModelForSeq2SeqLM.from_pretrained("johnlockejrr/marianmt_syr_voc")

# Input: consonantal Syriac text
text = "ܒܪܫܝܬ ܐܝܬܘܗܝ ܗܘܐ ܡܠܬܐ"

# Add language tag
input_text = f">>syr_cons<< {text}"

# Tokenize
inputs = tokenizer(input_text, return_tensors="pt", padding=True, truncation=True, max_length=512)

# Generate
outputs = model.generate(**inputs, max_length=512, num_beams=4, length_penalty=0.6)

# Decode
vocalized = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(vocalized)

Using the Pipeline

from transformers import pipeline

vocalizer = pipeline("text2text-generation", 
                     model="johnlockejrr/marianmt_syr_voc",
                     tokenizer="johnlockejrr/marianmt_syr_voc")

# Input text (consonantal)
text = "ܒܪܫܝܬ ܐܝܬܘܗܝ ܗܘܐ ܡܠܬܐ"
input_text = f">>syr_cons<< {text}"

# Vocalize
result = vocalizer(input_text, max_length=512, num_beams=4, length_penalty=0.6)
print(result[0]['generated_text'])

Text Normalization

The model expects input text to be normalized to NFC (Normalization Form Composed) Unicode format. The model automatically handles this, but for best results, ensure your input text is properly normalized:

import unicodedata

def normalize_text(text: str) -> str:
    """Normalize text to NFC format."""
    return unicodedata.normalize("NFC", text)

# Normalize input before processing
text = normalize_text("ܒܪܫܝܬ ܐܝܬܘܗܝ")

Input Cleaning

For optimal results, input text should contain only consonantal Syriac characters. The model is designed to work with raw consonantal text, but it can handle text with some punctuation. For best performance, remove vocalization marks from input text if present.

Generation Parameters

Recommended generation parameters:

• num_beams: 4 (beam search for better quality)
• length_penalty: 0.6 (encourages longer outputs)
• early_stopping: True
• max_length: 512 (matches training configuration)
• do_sample: False (deterministic generation)

Limitations and Bias

• Domain Specificity: This model is trained primarily on biblical Syriac texts. Performance may vary on other domains (e.g., modern Syriac, poetry, prose).
• Single Direction: The model only vocalizes consonantal text. It does not perform the reverse operation (removing vocalization).
• Length Constraints: Maximum input/output length is 512 tokens. Longer texts should be split into smaller segments.
• Character Accuracy: While BLEU and chrF scores are high, character-level accuracy is ~60%, meaning some diacritical marks may be missing or incorrect in complex cases.

Training Procedure

Training Infrastructure

• Hardware: GPU (CUDA)
• Training Time: ~12.7 hours
• Framework: Hugging Face Transformers
• Evaluation Frequency: Every 1,000 steps

Preprocessing

• Text normalized to NFC Unicode format
• Language tags (>>syr_cons<< and >>syr_voc<<) added to tokenizer vocabulary
• Tokenization using SentencePiece (inherited from base model)

Hyperparameters

{
  "learning_rate": 1e-5,
  "batch_size": 8,
  "gradient_accumulation_steps": 4,
  "num_epochs": 100,
  "max_input_length": 512,
  "max_target_length": 512,
  "warmup_steps": 1000,
  "weight_decay": 0.01,
  "eval_steps": 1000,
  "save_steps": 1000,
  "save_total_limit": 3
}

Evaluation

The model is evaluated using three metrics:

1. BLEU Score: Measures n-gram precision between generated and reference text
2. chrF Score: Character-level F-score, more lenient than BLEU
3. Character Accuracy: Exact character match percentage

Evaluation Results

Citation

If you use this model, please cite:

@misc{marianmt_syr_voc,
  title={MarianMT Syriac Vocalization Model},
  author={johnlockejrr},
  year={2025},
  howpublished={\url{https://huggingface.co/johnlockejrr/marianmt_syr_voc}},
  note={Fine-tuned from Helsinki-NLP/opus-mt-tc-bible-big-sem-en}
}

Acknowledgments

• Base Model: Helsinki-NLP/opus-mt-tc-bible-big-sem-en by the Helsinki NLP team
• Framework: Hugging Face Transformers
• Training Framework: MarianMT architecture

Model Card Contact

For questions, issues, or contributions, please open an issue on the model repository.

License

This model is released under the Apache 2.0 license, consistent with the base model.