Compound normalization model from CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models.

Usage

from transformers import pipeline

pipe = pipeline("text2text-generation", "benjamin/compoundpiece")

pipe("Hauswirtschaftslehre", max_length=32)
# [{'generated_text': 'Haus-Wirtschaft-Lehre'}]

Citation

@article{minixhofer2023compoundpiece,
  title={CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models},
  author={Minixhofer, Benjamin and Pfeiffer, Jonas and Vuli{\'c}, Ivan},
  journal={arXiv preprint arXiv:2305.14214},
  year={2023}
}

License

MIT