Aegis
Urdu span detector for personally identifiable information. This repository is the privacy-filter checkpoint. It is not the female TTS voice mahwizzzz/piper-voice-ur-aegis-female.
Purpose
Span-level detection of names, phone numbers, email addresses, addresses, dates, and account or CNIC-style identifiers in Urdu text, for assisted redaction. A demo that downloads this repository is mahwizzzz/de-identify.
Upstream attribution
finetune_summary.json records base_checkpoint as openai/privacy-filter. The same file records model_id as mahwizzzz/urdu-legal-privacy-filter, which is not this repository's id. config.json sets model_type to privacy_filter.
Span labels in config.json: O, private_person, private_phone, private_email, private_address, private_date, account_number.
Data provenance
Training data are mahwizzzz/uopf-sipar. That dataset repository is private, so the public cannot recheck it from the Hub.
Its dataset card tags the corpus synthetic and lists these train-split sources:
| Source | Records |
|---|---|
synthetic_multidomain_urdu_v1 |
32,000 |
llm_synthetic_realistic_urdu_v1 |
26,640 |
synthetic_legal_urdu_v1 |
15,396 |
synthetic_finance_urdu_v1 |
5,964 |
Those four counts sum to 80,000. The dataset card's split sizes, also repeated below, are 80,000 / 10,000 / 10,000. Generator model ids, prompts, and any human review are not in this model repository. The dataset card declares CC BY 4.0. This model card declares Apache-2.0. The model license does not relicense the dataset.
| Split | Records |
|---|---|
| Train | 80,000 |
| Validation | 10,000 |
| Test | 10,000 |
Train-domain counts stated on both cards: legal 21,986; finance 13,850; telecom 6,315; healthcare 5,410; travel 4,876; social_media 4,845; news_channel 4,818; government 3,965; ecommerce 3,923; insurance 3,382; education 3,319; hr_jobs 3,311.
Training
Figures below are copied from finetune_summary.json (updated_at 2026-05-22T08:26:21.708110Z). The checkpoint name in that file is uopf_sipar_prod_20260520_090709.
| Item | Value |
|---|---|
| Epochs | 3 (best epoch 2) |
| Batches | 40,000 |
| Tokens seen | 3,677,980 |
| Learning rate | 1e-5 |
| Batch size | 2 |
| Gradient accumulation | 1 |
| Weight decay | 0.01 |
| Max grad norm | 1.0 |
| Context window | 128000 |
| Tokenizer | o200k_base |
| Precision | bfloat16 |
| Device | CUDA (GPU model not named) |
| Final train loss | 0.000593 |
| Final validation loss | 0.001713 |
| Best validation loss | 0.001593 |
| Validation token accuracy | 0.9997 |
Evaluation
Two different records are in this repository. They are not the same result.
eval_metrics.json has an empty test object. Its validation block points at data/validation.jsonl and reports span-detection F1 0.9994091722694807, precision 0.9997770246164823, and recall 0.9990415905139749. Per-class span F1 in that file: private_person 0.9998982550742649, private_phone 1.0, private_email 0.9997391077484998, private_address 0.9978453383767306, private_date 1.0, account_number 0.9997052755673446. That file does not contain a base-model comparison.
The table below is retained from the previous card. It is not reproduced by eval_metrics.json. The card text described the metric as same-label overlap span F1 on a held-out test split, with exact span F1 requiring label and character offsets to match.
| Metric | Base openai/privacy-filter | Fine-tuned model | Absolute gain |
|---|---|---|---|
| Overlap detection F1 | 0.517 | 0.957 | +0.440 |
| Overlap precision | 0.497 | 0.997 | +0.500 |
| Overlap recall | 0.538 | 0.920 | +0.382 |
| Macro F1 | 0.468 | 0.931 | +0.463 |
| Exact span F1 | 0.021 | 0.406 | +0.385 |
| Entity | Base F1 | Fine-tuned F1 | Absolute gain |
|---|---|---|---|
| private_person | 0.509 | 0.966 | +0.457 |
| private_phone | 0.386 | 0.755 | +0.369 |
| private_email | 0.646 | 0.999 | +0.353 |
| private_address | 0.015 | 0.986 | +0.971 |
| private_date | 0.657 | 0.989 | +0.332 |
| account_number | 0.597 | 0.893 | +0.296 |
finetune_summary.json also records an evaluation_10k loss of 0.0015, 10,000 examples, 459,982 tokens, and 1004.19 inference tokens/sec. The script, split, and hardware for that timing are not named.
Limitations
- The public Hub copy of the training set cannot be inspected.
- The dataset card describes the text as synthetic. Performance on real documents is not reported in
eval_metrics.json. - The retained test table and the shipped validation JSON disagree. Treat them as separate, unreconciled records.
- The de-identify Space states that redaction is not a substitute for human review, and that Roman Urdu and mixed script are unsupported. Those statements are interface text, not rows in
eval_metrics.json. - The Space also says
account_numberis the weakest class. In the retained table above it is the lowest fine-tuned overlap F1. Ineval_metrics.jsonvalidation span F1 it is not the lowest class.
License
Apache-2.0, as declared in the metadata. A LICENSE file was not in the repository file list retrieved for this revision.
Usage
This repository does not include an inference script or a Transformers modeling file. config.json uses model_type: privacy_filter. The de-identify Space downloads mahwizzzz/aegis and decodes BIOES labels with a Viterbi routine implemented in that Space, not in this repository. ONNX exports are under onnx/. Their input contract is not documented here.
Citation
@misc{mahwiz2026urdu_privacy_filter,
title = {Aegis},
author = {Mahwiz Khalil},
year = {2026},
howpublished = {Hugging Face Model Hub},
url = {https://huggingface.co/mahwizzzz/aegis}
}