🛡️ Horizon-10M-Guard
A Tiny, Experimental LLM-Based Safety & Content Moderation Classifier
Built with 🖤 by Nebulixlabs
🧠 Model Overview
Horizon-10M-Guard is a lightweight, LLM-based text safety classification model created by applying Supervised Fine-Tuning (SFT) on top of Nebulixlabs/Horizon-10M-Base.
Designed as an ultra-compact guard model (~10M parameters), it evaluates input text across broad safety domains and classifies whether the content is safe or unsafe, while also predicting the corresponding harm sub-category. Despite its tiny size, it demonstrates how modern decoder architectures (RoPE + SwiGLU + RMSNorm) can be adapted into fast, edge-friendly content moderation classifiers.
🏗️ Architecture Details
| Component | Value |
|---|---|
| 🧬 Base Model | Nebulixlabs/Horizon-10M-Base |
| 🎯 Fine-Tuning Method | Supervised Fine-Tuning (SFT) |
| 🔢 Parameters | ~10.06M |
| 📚 Vocabulary size | 8,192 (Custom Byte-Level BPE) |
📐 Hidden size (d_model) |
320 |
| 🧱 Layers | 6 |
| 👁️ Attention heads | 5 (head_dim = 64) |
| 🔁 MLP hidden size | 864 (SwiGLU) |
| 🪟 Context length | 128 tokens |
| 🎯 Positional encoding | Rotary (RoPE) |
| ⚖️ Normalization | RMSNorm (pre-norm) |
📊 Evaluation & Performance
| Metric | Score | Notes |
|---|---|---|
| 🎯 Overall Accuracy | ~60% | Evaluated on held-out safety classification prompts |
Note: Achieving ~60% multi-category safety classification accuracy with only 10.06M parameters and a 128-token context window highlights the model's efficiency as an experimental research baseline, though it is not suitable for standalone production moderation.
🛡️ Supported Classification Scope
Horizon-10M-Guard supports general-purpose text safety classification across diverse topics and outputs:
- Safety Verdict:
safeorunsafe - Sub-Category Tag: Identifies the specific risk/topic category associated with the input text.
💻 How to Use
from transformers import AutoTokenizer, AutoModelForCausalLM
repo_id = "Nebulixlabs/Horizon-10M-Guard"
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo_id, trust_remote_code=True)
# Format your input prompt according to your SFT prompt template
prompt = "Classify the following text:\nText: How do I bake a chocolate cake?\nOutput:"
output = model.generate_text(
tokenizer,
prompt=prompt,
max_new_tokens=20,
temperature=0.1, # Low temperature recommended for deterministic classification
top_k=10,
top_p=0.9,
repetition_penalty=1.05
)
print(output)