echarlaix/bert-base-uncased-qqp-f87.8-d36-hybrid

🤗 Hugging Face sourcetext-classificationapache-2.0791 MBother✓ 2 checksumsupdated today

Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo echarlaix/bert-base-uncased-qqp-f87.8-d36-hybrid ./model-folder

Needs a seeder →

bert-base-uncased model fine-tuned on QQP

This model was created using the nn_pruning python library: the linear layers contains 36% of the original weights.

The model contains 50% of the original weights overall (the embeddings account for a significant part of the model, and they are not pruned by this method).

Fine-Pruning details

This model was fine-tuned from the HuggingFace model checkpoint on task, and distilled from the model textattack/bert-base-uncased-QQP. This model is case-insensitive: it does not make a difference between english and English.

A side-effect of block pruning is that some of the attention heads are completely removed: 54 heads were removed on a total of 144 (37.5%).

Details of the QQP dataset

Dataset	Split	# samples
QQP	train	364K
QQP	eval	40K

Results

Pytorch model file size: 377MB (original BERT: 420MB)

Metric	# Value
F1	87.87