ai-text-detector (weights)

These are the trained weights of github.com/lucadagati/ai-text-detector. It is a local detector of LLM-generated academic text that works per paragraph and per sentence.

The service is an OR ensemble of two DeBERTa-v3-base classifiers. Each one mean-pools the encoder and outputs a single logit, where higher means more machine-like:

A paragraph is flagged when either model exceeds its own threshold at 0.5% false positives, so the ensemble stays at 1% or less overall. model.json holds the thresholds, the sentence-window thresholds and the norms of the sentence-structure profile used for explanations.

Results

All results are at 1% false positives, on held-out papers (split by paper).

Claude models left out of training entirely are still detected at 95%. Small open-weight model families left out of training are detected at 59%. Full benchmark, protocol and limits: bench/RESULTS.md in the GitHub repository.

Use

The service in the GitHub repository downloads these weights on first start. To load one model directly:

from huggingface_hub import snapshot_download
from aidet.classifier import load_classifier          # from the GitHub repository
path = snapshot_download("Lukkinen/ai-text-detector", allow_patterns=["deberta-base-v2/*"])
model, tok = load_classifier(f"{path}/deberta-base-v2")

The classifier class (MeanPoolClassifier) is defined in aidet/classifier.py. It is not a stock AutoModelForSequenceClassification head.

Limits

• Trained and validated on English computer-science and engineering academic text.
• A score measures resemblance to LLM text. It is not proof of authorship.
• Detector-guided rewriting (iterating a rewrite until this detector scores it human) evades it.
• Isolated flagged paragraphs occur in about 8% of human papers.