A small (22.7M parameters) English text classifier for basic security filtering. It labels a message as one of four classes: safe, spam, phishing or prompt_injection.
| Metric | Score |
|---|---|
| Accuracy | 0.9865 |
| Macro F1 | 0.9850 |
| Weighted F1 | 0.9865 |
| Class | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| safe | 0.9906 | 0.9915 | 0.9910 | 2115 |
| spam | 0.9845 | 0.9878 | 0.9861 | 900 |
| phishing | 0.9732 | 0.9689 | 0.9710 | 900 |
| prompt_injection | 0.9922 | 0.9911 | 0.9917 | 900 |
| Metric | TinyGuard1-22M | TinyGuard1.1-22M | Fark |
|---|---|---|---|
| Accuracy | 0.9669 | 0.9865 | +0.0196 |
| Macro F1 | 0.9696 | 0.9850 | +0.0154 |
| Phishing Recall | 0.9255 | 0.9689 | +0.0434 |
| Safe F1 | 0.9698 | 0.9910 | +0.0212 |
| Spam F1 | 0.9758 | 0.9861 | +0.0103 |
| Phishing F1 | 0.9458 | 0.9710 | +0.0252 |
| Prompt Injection F1 | 0.9870 | 0.9917 | +0.0047 |
from transformers import pipeline
clf = pipeline("text-classification", model="Kuantum-ai1/TinyGuard1.1-22M")
print(clf("Congratulations! You won a free prize, click here to claim it."))
Same public datasets as v1.0, plus augmentation (typo, spacing, homoglyph, case) to balance classes.
| Class | Examples (after augmentation) |
|---|---|
| safe | 21148 |
| phishing | 9000 |
| spam | 9000 |
| prompt_injection | 9000 |
Fine-tuning with PyTorch on one T4 GPU: max length 320, batch size 64, learning rate 2e-5, 5 epochs, AdamW (weight decay 0.01), cosine schedule with 10% warmup, label smoothing 0.05, class weights, fp16. Best checkpoint selected by validation macro F1.
Apache-2.0
nreimers/MiniLM-L6-H384-uncased