ruT5-base-smart-home, CTranslate2 int8 Quantized

(int8) CTranslate2 version of ilyali034/ruT5-base-smart-home for fast CPU inference, for example on Raspberry Pi 5. See the original model card for the task, training data and limitations.

Квантизованная версия для быстрого запуска на процессоре (Raspberry Pi).

Usage

pip install ctranslate2 transformers sentencepiece huggingface_hub

from huggingface_hub import snapshot_download
import sys
path = snapshot_download("ilyali034/ruT5-base-smart-home-ct2_int8")
sys.path.insert(0, path)
from inference_ct2 import SmartHomeCT2

model = SmartHomeCT2(path, threads=4)
print(model.predict("включи свет на кухне"))