Multilingual Hate Speech Detector (base, 308M)

Detects hate speech, meaning attacks on people because of a protected characteristic, in English and about 50 other languages. It also says which group is targeted. Outputs (independent sigmoid probabilities):

• hate: the text attacks, demeans, dehumanises, stereotypes or calls for harm or exclusion of people because of their race or ethnicity, nationality or migration status, religion, gender, sexual orientation, gender identity, disability or age;
• target_race, target_religion, target_origin, target_gender, target_sexuality, target_age, target_disability: which group is attacked (meaningful when hate is high).

Ordinary insults, profanity, attacks on politicians, companies or sports fans, criticism of ideas, and counter-speech are not hate speech here, so it complements a general toxicity model such as our multilingual-toxicity. Built on mmBERT-base, Apache-2.0, ONNX for CPU and the browser (transformers.js) included.

• Flag with hate >= 0.5, or tune the threshold on your own data. onnx/model_quantized.onnx (int8 embeddings, 641 MB): probabilities differ from fp32 by 0.0012 on average over 448 test texts.

Usage

from transformers import pipeline

clf = pipeline("text-classification", model="Horizon-Labs/multilingual-hate-speech-base", top_k=None)
print(clf("Die sind alle Ungeziefer, schmeißt sie raus."))
# [[{'label': 'hate', 'score': ...}, {'label': 'target_origin', 'score': ...}, ...]]

transformers.js:

import { pipeline } from "@huggingface/transformers";
const clf = await pipeline("text-classification", "Horizon-Labs/multilingual-hate-speech-base", { dtype: "q8" });
console.log(await clf("Immigrants are vermin.", { top_k: null }));

Evaluation

Every model is scored the same way (code/hate/eval_hate.py): ROC AUC of its hate probability, and F1 at 0.5. No test item was used for training.

• Multilingual HateCheck (mteb/multi-hatecheck, CC-BY-4.0): 40,310 functional test cases in 11 languages, including hard non-hateful cases (counter-speech, negation, abuse of individuals or objects, neutral mentions of groups).
• Held-out real comments (no text from these sets was used in training): Stormfront forum sentences (hate_speech18, CC-BY-SA-3.0), ETHOS binary (YouTube and Reddit), OLID-BR test (Brazilian Portuguese; racism, LGBTQphobia, religious intolerance, sexism or xenophobia count as hate) and K-MHaS (Korean news comments, 5,000; attacks on protected groups vs not hate speech).
• ¹ In-domain: Measuring Hate Speech (6,000 comments, hateful = the authors' score > 0.5) and the HateXplain test split (offensive-but-not-hate counts as negative). Other texts from these two datasets (not the test items, and not their labels) are in our training data.

• The table shows the released checkpoint. Means over 2 training seeds (HateCheck .866 / .867, held-out real comments .841 / .841, Measuring Hate Speech + HateXplain + K-MHaS .804 / .805).
• The English baselines do well on English but are near chance on the other HateCheck languages. Among models not trained on HateCheck, this model leads in 10 of 11 HateCheck languages (Arabic, Mandarin, German, French, Hindi, Italian, Dutch, Polish, Portuguese, Spanish). On the held-out real-comment mean, models ahead of it: none (teacher excluded). The XLM-R-large model's near-perfect HateCheck score comes with no training-data documentation; its real-comment scores are the comparable evidence.
• Caveat: our synthetic training data deliberately includes counter-speech, quoted hate, offensive-but-not-hateful text and implicit hate, which HateCheck also tests (no HateCheck items or templates were used). The held-out real comments are the more independent test.

Held-out real comments by set (AUC):

Per HateCheck language:

HateCheck by test type (mean over languages): hateful cases detected 0.67; non-hateful cases correctly left alone:

Target groups (Measuring Hate Speech, hateful comments, AUC of each target_* output vs the annotated target): race 0.89, religion 0.96, origin 0.92, gender 0.90, sexuality 0.92, age 0.83, disability 0.94.

Training

• Texts: 119,983 comments from Civil Comments (CC0) and their machine translations from Horizon-Labs/civil-comments-multilingual (half selected for identity-related attacks), plus 85,732 texts written natively in 52 languages by Qwen3.8-27B (Apache-2.0) for this purpose: hateful texts against groups of 7 protected categories (explicit and implicit or coded), non-hateful texts mentioning those groups (neutral, news-style, counter-speech, quoted hate), and offensive texts that are not hate speech. Plus 50,793 real social-media posts: Measuring Hate Speech comments outside our 6,000-comment test sample, and the HateXplain train and validation splits (CC-BY-4.0), counted twice. Only their text was used; the labels came from the teacher.
• Labels: Qwen3.8-27B (prompted with the definition above) answered one yes/no question per output for every text; the model learns these probabilities (soft labels, binary cross-entropy). The checkpoint was chosen on 12,881 held-out teacher-labelled texts.
• mmBERT-base, max length 256 tokens. Code: code/ in this repository.

Limitations

• What counts as hate speech depends on law, platform policy and culture. This model follows one definition (above). Check it against your own policy and data before relying on it.
• Labels come from an LLM teacher, not human annotators, and its errors carry over. The teacher's own scores are in the table.
• Weaker on implicit hate, on Arabic and Hindi (see the per-language table), and on languages and dialects that are rare in the data.
• Do not use it as the only basis for decisions about people. Keep a human in the loop for enforcement.