Document Sensitivity Tier Classifier
Reads a short description of a document and suggests a sensitivity tier: public, internal, confidential or restricted.
Every name, number and sentence it was trained on is made up. Nothing comes from a real person, customer or business.
Related tool: document-sensitivity-keyword-sorter (a Space that runs in your browser).
Why it exists
Before data leaves a company, someone sorts what it holds by how sensitive it is. This model gives that sort a first draft.
Use it
from transformers import pipeline
classify = pipeline("text-classification", model="aitrainingdatabroker/document-sensitivity-tier-classifier", top_k=3)
print(classify("Customer database export with emails, phone numbers and home addresses."))
How it was trained
Fine-tuned from distilroberta-base for 6 epochs on one RTX 3070. Training data: 305 rows from synthetic-document-sensitivity-tiers, all invented, with test rows built from templates the model never saw in training. The scripts that built the data and trained the model are in source/; the full scores are in scores.json.
Test scores
On 77 held-out rows: accuracy 0.831, macro F1 0.823.
| Label | F1 | Count |
|---|---|---|
confidential |
0.857 | 19 |
internal |
0.816 | 20 |
public |
0.974 | 20 |
restricted |
0.645 | 18 |
Limits
- Trained only on invented text from templates, with test sentences built from templates it never saw. Real text varies far more.
- Tiers differ between companies and laws. A model's tier is a suggestion; the owner of the data sets the tier.
- This model is not legal advice and not a certification. A person should review its output before anything is shared or decided, and your counsel decides what a rule requires.
Read more
- De-identification before data is licensed
- Privacy rules by region
- Can you license SOPs and knowledge bases?
License and credit
Apache 2.0. Please credit AI Training Data Broker and link to https://aitrainingdatabroker.com/ when you share or build on it.
About
AI Training Data Broker is a broker and technical advisor for companies that sell or license their data to AI labs. It guides the technical side, including export, format, de-identification and compliance, and publishes free guides, checklists and tools.