سِتر · Sitr — Arabic & Gulf PII Detection

A token classifier for personal data in Arabic, adapted to Saudi, Gulf and Levantine formats. Built on openai/privacy-filter and trained on mabahboh/sitr-arabic-pii.

What it detects

43 fine-grained classes, tagged BIO over 87 tags:

access_token, account_number, api_key, bahrain_cpr, boundary_number, building_number, card_cvv, card_pin, commercial_registration, contract_number, credit_card, db_connection, deed_number, emirates_id, employee_id, gulf_iban, gulf_phone, insurance_policy, iqama, kuwait_civil_id, medical_record_number, medicine_name, national_address, national_id, oman_civil_id, otp_code, passport_number, password, postal_code, private_address, private_date, private_email, private_phone, qatar_qid, short_address, spoken_iqama, spoken_national_id, spoken_otp, spoken_phone, syria_national_id, syria_phone, vehicle_plate, visitor_number

These fold onto 8 base classes - account_number, national_id, other, private_address, private_date, private_email, private_phone, secret - and label_scheme.json ships alongside the weights so the collapse can be reproduced exactly.

Results

Entity-level scores on the held-out test split: a span counts only when its boundaries and its class both match. Splits are partitioned by document family, so no near-duplicate of a test row appears in training.

Level	Precision	Recall	F1
Fine-grained (43 classes)	0.629	0.776	0.694
Base classes (8)	0.788	0.852	0.818

The base-class row is the number that reflects redaction: confusing two sibling identifier types still masks the span. The gap between the rows is the cost of sub-type confusion alone.

Per class

Class	Precision	Recall	F1	Support
`access_token` *	0.607	0.872	0.716	39
`account_number`	0.917	0.981	0.948	213
`api_key` *	0.519	0.818	0.635	33
`bahrain_cpr` *	0.059	0.091	0.071	11
`boundary_number` *	0.310	0.333	0.321	39
`building_number` *	0.435	0.612	0.508	49
`card_cvv` *	0.565	0.722	0.634	18
`card_pin` *	0.286	0.118	0.167	17
`commercial_registration`	0.526	0.714	0.606	84
`contract_number`	0.553	0.727	0.628	150
`credit_card`	0.820	0.934	0.874	122
`db_connection` *	0.667	0.846	0.746	26
`deed_number`	0.435	0.709	0.539	103
`emirates_id` *	1.000	1.000	1.000	18
`employee_id`	0.774	0.827	0.800	249
`gulf_iban` *	0.512	0.750	0.609	28
`gulf_phone` *	0.093	0.154	0.116	26
`insurance_policy`	0.847	0.936	0.890	172
`iqama`	0.448	0.681	0.540	188
`kuwait_civil_id` *	0.125	0.125	0.125	8
`medical_record_number`	0.490	0.708	0.580	144
`medicine_name`	0.000	0.000	0.000	57
`national_address`	0.916	0.974	0.944	156
`national_id`	0.751	0.890	0.815	390
`oman_civil_id` *	0.286	0.429	0.343	14
`otp_code` *	0.197	0.293	0.235	41
`passport_number`	0.716	0.817	0.763	71
`password`	0.653	0.790	0.715	62
`postal_code`	0.484	0.754	0.590	61
`private_address`	0.188	0.032	0.055	93
`private_date`	0.882	0.940	0.910	215
`private_email`	0.925	0.966	0.945	293
`private_phone`	0.744	0.893	0.812	488
`qatar_qid` *	0.324	0.579	0.415	19
`short_address`	0.901	0.924	0.912	79
`spoken_iqama` *	0.010	0.059	0.017	17
`spoken_national_id` *	0.076	0.250	0.117	32
`spoken_otp` *	0.133	0.222	0.167	18
`spoken_phone` *	0.310	0.481	0.377	27
`syria_national_id` *	0.154	0.222	0.182	9
`syria_phone` *	0.025	0.045	0.032	22
`vehicle_plate`	0.482	0.675	0.562	80
`visitor_number` *	0.091	0.231	0.130	39

* fewer than 50 test spans - indicative only, not a measurement.

Read recall first. A missed span is a personal identifier reaching an external model and cannot be undone; a false positive is an over-redacted word. The two errors are not equally costly.

Usage

from transformers import pipeline

nlp = pipeline("token-classification", model="mabahboh/sitr-privacy-filter-ar",
               aggregation_strategy="simple")

nlp("هذا رقم هويتي 1429096553 وجوالي 0551234567")

Training


Method	LoRA (r=16, alpha=32, dropout=0.05), merged into the base weights
Adapted modules	k_proj, o_proj, q_proj, v_proj
Learning rate	0.0002
Epochs	3
Effective batch	32
Max sequence length	512 (stride 128)
Auxiliary base-class loss	weight 0.3

Training used a single head over the fine classes, with the base-class signal marginalised out of the same logits rather than added as a second head - the probability of a base class is the sum of the probabilities of every fine class folding into it. One distribution, so the two levels cannot contradict each other.

The adapter was applied to attention projections only. On a Mixture-of-Experts backbone this matters: routing weights are left untouched, so expert assignment is unchanged by training or by the merge.

Limitations

Not a compliance guarantee. This reduces exposure; it does not certify that a system is anonymous or PDPL-compliant.
Person names are not covered - private_person is absent from the training data, and names are where Arabic PII models fail hardest.
Many classes are not separable by span alone. Ten bare digits is simultaneously a national ID, an iqama, a phone, a commercial registration and more; only context separates them. Classes marked * above have too little test support to measure.
Evaluated on the dataset's own test split. Performance on live traffic will be lower.

Licence

Apache-2.0.

mabahboh/sitr-privacy-filter-ar-v1