中文Sign up

softcatala/translate-deu-cat

🤗 Hugging Face sourcetranslationmit196 MBother✓ 2 checksumsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo softcatala/translate-deu-cat ./model-folder
Needs a seeder →

Introduction

German - Catalan translation model for OpenNMT. These are the same models that we have in production at https://www.softcatala.org/traductor/. The models are quantified for low latency.

Usage

Install the necessary dependencies:

pip3 install ctranslate2 pyonmttok

Simple tokenization & translation using Python:

import ctranslate2
import pyonmttok
from huggingface_hub import snapshot_download
model_dir = snapshot_download(repo_id="softcatala/translate-deu-cat", revision="main")

tokenizer=pyonmttok.Tokenizer(mode="none", sp_model_path = model_dir + "/sp_m.model")
tokenized=tokenizer.tokenize("Hallo Freunde")

translator = ctranslate2.Translator(model_dir)
translated = translator.translate_batch([tokenized[0]])
print(tokenizer.detokenize(translated[0][0]['tokens']))

Benchmarks

testset BLEU
test dataset (from train/dev/test) 34.8
Flores200 dataset 28.9

Additional information