jackaduma/SecRoBERTa

🤗 Hugging Face sourcefill-maskapache-2.084M params336 MBsafetensors✓ 2 checksumsupdated today

Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo jackaduma/SecRoBERTa ./model-folder

Needs a seeder →

SecRoBERTa

This is the pretrained model presented in SecBERT: A Pretrained Language Model for Cyber Security Text, which is a SecRoBERTa model trained on cyber security text.

The training corpus was papers taken from

SecRoBERTa has its own wordpiece vocabulary (secvocab) that's built to best match the training corpus.

We trained SecBERT and SecRoBERTa versions.

Available models include:

Fill Mask

We proposed to build language model which work on cyber security text, as result, it can improve downstream tasks (NER, Text Classification, Semantic Understand, Q&A) in Cyber Security Domain.

First, as below shows Fill-Mask pipeline in Google Bert, AllenAI SciBert and our SecBERT .

The original repo can be found here.