project-creation-quality-distilbert-v1

Version 1 of the MyVillage V3 project-creation quality classifier. It is DistilBERT (distilbert/distilbert-base-uncased, ~67M parameters), fine-tuned as a 12-label multilabel classifier over the semantic and hybrid failure codes of taxonomy v3.0.0.

It is meant to run inside the hybrid system. The bundled v3cls/ package adds the deterministic validator (7 rule-based codes and the hybrid rules) and the decision engine. The system returns exactly:

{"decision": "ACCEPT" | "REJECT", "failed_requirements": ["CODE", ...]}

Usage

from huggingface_hub import snapshot_download
import sys
path = snapshot_download("mjpsm/project-creation-quality-distilbert-v1")
sys.path.insert(0, path)
from v3cls.inference import V3Classifier
clf = V3Classifier(path)
clf.classify(prompt="...", title="...", description="...")

Data

There are 1,000 synthetic and golden-derived records (300 accept / 490 single-violation / 210 multi-violation), split 800/100/100 by concept. Thresholds were calibrated on validation (v3_thresholds.json). All metrics below are on the untouched test split.

Test results (full hybrid system)

Test results (model labels, calibrated thresholds)

Limitations

The test split is small: 100 records, with 2–9 positives per label. Thresholds come from a 100-record validation split. The training data is mostly synthetic contrastive edits of compliant bases. Review the outputs before relying on this model for production gating.