MiniLM Content Guard - 3 Class

A lightweight content moderation model that classifies text into safe, toxic, or spam. Built on MiniLMv2-L6-H384 and fine-tuned with focal loss for robust handling of hard examples. This model is in ONNX format and optimized for CPU inference.

suggested threshold for considering results as valid: 0.9 (if less than 0.9 confidence, there can be a risk that prediction is wrong)

Labels

Usage

ONNX Runtime

from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer, AutoConfig
import torch

model_name = "navodPeiris/minilm-toxic-spam-classifier"

tokenizer = AutoTokenizer.from_pretrained(model_name)
config = AutoConfig.from_pretrained(model_name)
model = ORTModelForSequenceClassification.from_pretrained(model_name)

text = "look like garbage!"

inputs = tokenizer(text, return_tensors="pt")

outputs = model(**inputs)

# Convert logits → probabilities
probs = torch.softmax(outputs.logits, dim=-1)

# Get predicted class
pred_id = torch.argmax(probs, dim=-1).item()

label = config.id2label[pred_id]
confidence = probs[0][pred_id].item()

print(label, f"{confidence}")

Transformers.js

import { pipeline } from "@huggingface/transformers";

const pipe = await pipeline(
  "text-classification",
  "navodPeiris/minilm-toxic-spam-classifier",
);

const res = await pipe("this code is uglier than u ugghh");
console.log("res:", res);

Performance

Evaluated on a held-out test set of 13,123 samples:

            precision  recall   f1-score   support

safe          0.98      0.96      0.97      4332
toxic         0.90      0.96      0.93      1626
spam          0.99      0.97      0.98      1157

accuracy                          0.96      7115
macro avg     0.95      0.96      0.96      7115
weighted avg  0.96      0.96      0.96      7115

Training Details

Architecture

• Base model: MiniLMv2-L6-H384-distilled-from-BERT-Large (6 layers, 384 hidden dim, ~22M params)
• Task head: Linear classification head (3 classes)
• Max sequence length: 512 tokens

Training Data

The model was trained on a combined dataset from multiple sources:

Hyperparameters

• Epochs: 5
• Batch size: 16 (train) / 32 (eval)
• Learning rate: 3e-5
• Weight decay: 0.01
• Loss: Focal loss (gamma=2) for better handling of hard/borderline examples
• Early stopping: Enabled on F1 macro

ONNX Export

An ONNX version of the model is included for fast CPU inference:

• Opset version: 18
• Dynamic axes: batch size and sequence length
• Constant folding: Enabled

Limitations

• English-only — not tested on other languages
• May struggle with subtle or implicit toxicity where the language closely resembles negative sentiment (e.g., strong product complaints vs. personal attacks)
• Not designed for nuanced content policy enforcement — best used as a first-pass filter

License

Apache 2.0