This fine-tuned model is licensed under the Apache License 2.0. It is based on
distilbert/distilbert-base-uncased,
which is also licensed under Apache-2.0. See the
Apache License 2.0 for the
license terms.
This is a fine-tuned DistilBERT sequence classifier for categorizing
job-search emails into five workflow labels. It uses the
distilbert/distilbert-base-uncased
base model.
DistilBertForSequenceClassificationapplication_confirmation, assessment,
interview_invitation, offer, and rejectionSubject: {subject}\nEmail: {body}from transformers import pipeline
model_id = "mhshahzad/distilbert-job-email-classifier_v3"
classifier = pipeline("text-classification", model=model_id, tokenizer=model_id)
text = (
"Subject: Interview invitation\n"
"Email: We would like to invite you to interview next week."
)
results = classifier(text, truncation=True, max_length=256, top_k=None)
print(results)
Scores are model probabilities, not calibrated confidence estimates.
This model is an experimental tool for organizing an individual's own job-search emails. Review every prediction before taking action.
Do not use it for hiring, candidate screening, employment decisions, or other consequential use. It is not a general-purpose email classifier and has not been validated across different languages, organizations, or recruiting systems.
The v3 run fine-tuned distilbert-base-uncased on the previous experimental
training sample, using subject and body together. Run notes indicate that the
sample included 50 synthetic offer examples; synthetic examples were used for
training only. V3 used a maximum sequence length of 256, learning rate 2e-5,
seed 42, and eight configured epochs. The checkpoint was selected using
validation macro-F1; its best validation macro-F1 was 0.8580.
The evaluation report records dataset revision
28a2b423bf0be636ae0736a35cc96c4f290a8277 from the private dataset
repository
mhshahzad/job_application_emails_clean_training_dataset.
Dataset access is separately controlled, and dataset contents are not
redistributed with this model card.
The latest reported DistilBERT experiment (v3) used the email subject and body together. The v3 evaluation covers 973 examples. Labels are project reference labels and were not independently adjudicated.
| Metric | DistilBERT v3 |
|---|---|
| Accuracy | 0.9681 |
| Macro-F1 | 0.8673 |
| Weighted-F1 | 0.9677 |
| Validation macro-F1 | 0.8580 |
| Mean latency | 1.90 ms/email |
Per-class v3 test results:
| Category | Precision | Recall | F1-score | Support |
|---|---|---|---|---|
| Application confirmation | 0.9726 | 0.9844 | 0.9785 | 577 |
| Assessment | 0.9091 | 0.7692 | 0.8333 | 13 |
| Interview invitation | 0.9318 | 0.8542 | 0.8913 | 48 |
| Offer | 1.0000 | 0.5000 | 0.6667 | 2 |
| Rejection | 0.9670 | 0.9670 | 0.9670 | 333 |
Earlier reported experiments provide context:
| Experiment | Input | Accuracy | Macro-F1 | Weighted-F1 |
|---|---|---|---|---|
| DistilBERT v1 | Body only | 0.9620 | 0.8444 | 0.9613 |
| DistilBERT v2-2, expanded synthetic training data | Subject + body | 0.9496 | 0.7094 | 0.9489 |
| DistilBERT v3, latest | Subject + body | 0.9681 | 0.8673 | 0.9677 |
| BERT-base, original benchmark | — | 0.9630 | 0.8586 | 0.9600 |
V2-2 added synthetic offer and assessment examples to training. V3 used the earlier training sample with subject and body together. The v2-2 and v3 reports reference different dataset repositories/revisions. Their results are experiment snapshots, not a controlled, paired test of synthetic-data augmentation or input representation. The earlier BERT-base run emitted missing/unexpected LayerNorm-key warnings during checkpoint loading, so its reported score remains provisional. See the project README for the aggregate evaluation dashboard.
These are single-run results, not a multi-seed benchmark. The test set contains only two offer examples and 13 assessment examples, making those class metrics highly uncertain. Conversation-level separation cannot be confirmed because thread identifiers were unavailable.
Job-search emails can contain personal and sensitive information. Do not send email text to services or share it without appropriate authorization. The model card reports aggregate metrics and does not include email content or row-level predictions.
If you use or discuss this model, cite the model repository and identify the v3 evaluation as a single experimental result. No publication is associated with this model card.