echarlaix/distilgpt2-openvino-4bit

🤗 Hugging Face sourcetext-generationapache-2.0322 MBother✓ 1 checksumupdated today

Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo echarlaix/distilgpt2-openvino-4bit ./model-folder

Needs a seeder →

This model is a quantized version of echarlaix/distilgpt2-openvino and is converted to the OpenVINO format. This model was obtained via the nncf-quantization space with optimum-intel.

First make sure you have optimum-intel installed:

pip install optimum[openvino]

To load your model you can do as follows:

from optimum.intel import OVModelForCausalLM

model_id = "echarlaix/distilgpt2-openvino-4bit"
model = OVModelForCausalLM.from_pretrained(model_id)