Intel/t5-small-xsum-int8-dynamic-inc

🤗 Hugging Face sourceapache-2.0162 MBother✓ 2 checksumsupdated today

Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo Intel/t5-small-xsum-int8-dynamic-inc ./model-folder

Needs a seeder →

INT8 T5 small finetuned on XSum

Post-training dynamic quantization

This is an INT8 PyTorch model quantized with huggingface/optimum-intel through the usage of Intel® Neural Compressor.

The original fp32 model comes from the fine-tuned model adasnew/t5-small-xsum.

The linear modules lm.head, fall back to fp32 for less than 1% relative accuracy loss.

Evaluation result

	INT8	FP32
Accuracy (eval-rouge1)	29.9008	29.9592
Model size	154M	242M

Load with optimum:

from optimum.intel import INCModelForSeq2SeqLM

model_id = "Intel/t5-small-xsum-int8-dynamic-inc"
int8_model = INCModelForSeq2SeqLM.from_pretrained(model_id)