Browser-ready ONNX sentence-quality classifiers for AIGC Chinese text, fine-tuned from BAAI/bge-base-zh-v1.5.
Each sentence is scored in isolation into six classes:
| # | 标签 | 含义 |
|---|---|---|
| 1 | 烂完了 | 八股味重、语义崩坏,单看这句就要重写 |
| 2 | 小改 | 有明显问题,修一修还能用 |
| 3 | 平庸 | 能读,但平庸无趣,缺少打磨 |
| 4 | 还可以 | 读起来挺顺,可以保留 |
| 5 | 优秀 | 文笔出色,值得保留 |
| 6 | 跳过 | 无需判断(章节号、标记、结构化信息、非正文内容) |
Class 6 is a separate "skip" head, not a point on the 1–5 quality scale.
expected_quality is computed over classes 1–5 after renormalising.
| Path | Model | Size | Precision | Spearman | MAE |
|---|---|---|---|---|---|
onnx/8l-int8/model.onnx |
8L KD distilled | 70.7 MiB | INT8 | 0.6291 | 0.7481 |
onnx/12l-fp32/model.onnx |
12L interpolated | 388.1 MiB | FP32 | 0.6406 | — |
Both are browser-native: the graph uses only ops that onnxruntime-web's WASM
backend executes efficiently. The INT8 graph uses DynamicQuantizeLinear +
MatMulInteger, so it stays fast without a GPU.
Use 8l-int8 by default. It is 5.5× smaller and only 0.012 Spearman behind,
which matters far more for a web download than the accuracy delta.
Score and triage Chinese prose sentence by sentence — reviewing long-form AIGC output, finding sentences a human should rewrite, and filtering non-content.
Not suitable for: attributing text to a specific author or model, or as proof that text is machine-generated. The classifier was trained on human quality votes; it scores quality, and correlated style is not evidence of provenance.
The model takes input_ids and attention_mask (int64) and returns a
[batch, 6] logit tensor. Apply the fitted temperature before softmax —
it is required for calibrated probabilities.
import numpy as np
import onnxruntime as ort
from transformers import AutoTokenizer
REPO = "GoForceX/bge-roleplay-judge-onnx"
TEMPERATURE = 2.3511865550320645 # 8l-int8; 2.337502972054673 for 12l-fp32
LABELS = ["烂完了", "小改", "平庸", "还可以", "优秀", "跳过"]
ort_session = ort.InferenceSession(
"onnx/8l-int8/model.onnx", providers=["CPUExecutionProvider"]
)
tokenizer = AutoTokenizer.from_pretrained("BAAI/bge-base-zh-v1.5")
sentence = "「上课?上什么课?我连他们讲话都听不懂,上哪门子课咯?」"
enc = tokenizer(sentence, truncation=True, max_length=512, return_tensors="np")
logits = ort_session.run(
["logits"],
{
"input_ids": enc["input_ids"].astype(np.int64),
"attention_mask": enc["attention_mask"].astype(np.int64),
},
)[0][0]
z = logits - logits.max()
probs = np.exp(z / TEMPERATURE) / np.exp(z / TEMPERATURE).sum()
quality = probs[:5] / probs[:5].sum()
expected = float((quality * np.arange(1, 6)).sum())
print(LABELS[int(probs.argmax())], round(expected, 3))
The INT8 graph uses DynamicQuantizeLinear, whose activation scale is derived from
the whole input tensor. Padding therefore perturbs results slightly (measured over
240 sentences: probabilities differ by 0.007 on average, expected quality by 0.045).
Score one sentence at a time when you need to reproduce the published metrics;
batch only for throughput.
Both models start from BAAI/bge-base-zh-v1.5 and share the same head:
last_hidden_state[:, 0] (CLS)
-> LayerNorm + Dropout(0.1)
-> Linear(768, 6)
-> softmax over 6 classes
Objective: human 6-way soft-vote cross-entropy + 0.05 × ordinal loss over classes 1–5. One six-way distribution is used rather than a separate binary skip head, which won an internal ablation. Labels come from 8-vote-per-sentence human annotation.
kd-a025-8l-refine767): 6-layer prune of the interpolated 12-layer teacher
(layer map [0, 2, 4, 6, 8, 11]), then 8-layer knowledge-distilled refinement
(KD weight 0.8, hidden 0.04, low-LR tail update).interp-140-697-a025-v2): weight-space interpolation of an early and a
late fine-tuning checkpoint, α = 0.25.Evaluation is on a 595-sentence held-out split, ranked against human mean quality (Spearman) with expected-quality MAE.
manifest.json lists both models with size, SHA-256, fitted temperature and metrics;
tokenizer.json is mirrored here for reference. The companion web app reads the
manifest and streams weights from this repo, so no .onnx is bundled with the site.
onnx/12l-fp32/model.onnx 407 MBonnx/8l-int8/model.onnx 74 MBBAAI/bge-base-zh-v1.5