feishengmt-GGUF
GGUF Q4_K_M (1.1 GB) of hsilvosa/feishengmt, a fine-tune of tencent/Hy-MT2-1.8B for Chinese-to-English translation of wuxia and xianxia fiction, mainly English and adapted to answer also in French, Portuguese, Spanish, Indonesian, Japanese and Korean. Version 1.2; not endorsed by Tencent. Read the main model card for the training, the evaluation and the limits, or the blog post for the full development.
Quality
Quantization changes the scores little: on the external test of the main card (the whole window as one prompt, greedy decoding, one run):
| bf16 (merged checkpoint, 3.6 GB) | GGUF Q4_K_M (this file) | |
|---|---|---|
| chrF | 53.0 | 53.0 |
| BLEU | 22.6 | 22.7 |
| Same paragraphs as the source | 98.9 % | 98.9 % |
| Chinese left in the output | 0.4 % | 1.5 % |
Speed on an RTX 3060 with llama.cpp (the same windows for every model): 1.73 s per window with one request, 1.06 s with four in parallel (Index-Translate-2B Q4_K_M on the same windows: 2.02 s and 1.17 s). The other languages were not measured on the GGUF.
Use it
The pipeline/ folder contains the glossary and the translation CLI. It cuts the text into windows of paragraphs and
checks the answer, so use it for English.
huggingface-cli download hsilvosa/feishengmt-GGUF --local-dir feishengmt
llama-server -m feishengmt/feishengmt-Q4_K_M.gguf -c 8192 -np 4 --jinja --port 8080
# in another terminal
cd feishengmt/pipeline
python -m webnovel_mt.translate chapter_zh.txt -o chapter_en.txt --url http://127.0.0.1:8080
-np 4 with -c 8192 gives each of the four parallel slots 2,048 tokens of context: enough for a window of up to 400
characters, the glossary lines and the answer. Tested with llama-server only (build 10615); other runtimes such as
Ollama or LM Studio were not tested. With a plain chat interface use the prompt of the main model card, where the way to
ask for another language is also described.
License and attribution
Apache-2.0. Based on Hy-MT2 by Tencent (Apache-2.0); see NOTICE. This is a modified, quantized version of the original
model.