newmean-pii-medium

A small multilingual model that finds personal information (PII) in text so it can be masked in the browser, before the text leaves the device. It is the medium tier (17.7M parameters, 11.3 MB download) of the models behind the JavaScript SDK @newmean-ai/pii. Other tiers: newmean-pii-large, newmean-pii-small, newmean-pii-nano.

• 54 languages, 11 labels at the character level: PERSON, EMAIL, PHONE, ADDRESS, ID_NUMBER, FINANCIAL, ORG, USERNAME, URL, DATE_OF_BIRTH, SECRET; persons and URLs carry a level (1 = directly identifying, 2 = indirect). IP addresses are learned as ID_NUMBER. The SDK has an optional pattern layer (e-mail, URL, card/account, phone, Korean resident number), off by default.
• Built for real input: forms, chats, spreadsheet cells, logs, JSON, OCR text, e-mails, speech transcripts.
• Apache-2.0, free for any use including commercial use; ONNX with 4-bit weights; runs in WebAssembly (onnxruntime-web or the SDK's own built-in engine).

Use

Try it in your browser first: pii.newmean.ai.

npm install @newmean-ai/pii

import { createRedactor } from "@newmean-ai/pii";

const redactor = await createRedactor({ size: "medium" });   // downloads this repository's files once, checks their SHA-256
const { text, spans } = await redactor.redact("Hi, I'm Kim Minji, call me at 010-1234-5678.");

SDK 1.2 pins revision v1.1.0 of this repository (SDK 1.0 pins v1.0.0). The files can also be used directly with onnxruntime: the model takes input_ids and attention_mask (windows of 256 tokens, stride 64) and returns BIO logits for the tags in config.json.

Results

Model only (no pattern layer), default file, onnxruntime on CPU. Miss = share of PII characters left unmasked (lower is better; what matters most for masking); over = share of non-PII characters masked.

Speed on a Colab L4 VM's CPU (Xeon 2.20 GHz, onnxruntime): 88 ms per text with one thread, 23 ms with 12 threads, on the public test's texts. The public test of newmean-pii-dataset is generated with the same design and guideline as the training data, so these models have an in-distribution advantage there; the external sets were never used for training. On the public test the closest of the eight public detectors we compared (gliner-pii-edge, 176 MB) misses 19.6%; the leaderboard scores them weekly on its public and private test. The full comparison, per-label and per-language results are in the technical report.

Short texts and survey files (v1.1). On a private set of 20,691 short items in 54 languages (column titles, answer options, single words, names and values written alone) and 228 survey CSV files: masks 7.8% of the PII-free items (v1.0: 30.6%) and 5.9% of the PII-free cells when a survey file is checked cell by cell (v1.0: 22.5%); finds 97.5% of names written alone. The set is generated (by a different model from the training data) and not published.

With SDK 1.2. The numbers above move by at most 0.04 points, because the SDK only changes texts with emoji or keycaps. Emoji are taken out of the text the model reads: texts with an emoji masked fall from 14.89% (model only) to 0.78%. Table mode with context: "header" misses 1.93% of the PII characters of 556 survey and form exports in 54 languages (table mode alone: 7.93%) and masks 11.66% of their PII-free cells. Both sets are private and synthetic.

Model

6 layers, embeddings rank 192; 17.7M parameters. All four tiers start from mMiniLMv2-L6-H384 with the XLM-R vocabulary pruned to the 35,298 pieces that occur in the training data, and are distilled from two larger private teacher models (mDeBERTa-v3-base and XLM-R-large, fine-tuned for this task). Training data: about 1.31 million synthetic texts in 54 languages generated with Claude models from a written labelling guideline (v1.1 added 240,498 short texts: titles, answer options, single words, lone names and values, survey rows); no real personal data and no external dataset was used for training.

Limitations

• No detector finds everything: use the result to reduce what is sent, not as a guarantee of anonymity.
• Trained on synthetic text; rare formats, very long documents and languages outside the 54 may be weaker.
• Context decides some labels (an organisation is masked only when a person belongs to it; dates only when they are birth dates), so results depend on the surrounding text. sensitivity in the SDK trades misses for over-masking.
• Survey files checked cell by cell: use newmean-pii-large or sensitivity: -0.5; at the default, smaller tiers can mask a lone gender word ("남성", "Male").

Changelog

• v1.1.0 (2026-11-01, published with SDK 1.2.0 on 2026-11-02): short texts, table cells and survey exports. PII-free short items masked: 30.6% → 7.8%. Same architecture, tokenizer and sizes. KDPII miss 9.8% → 11.5% (higher than v1.0).
• v1.0.0 (2026-10-31): first release.

License and attribution

Apache-2.0 (LICENSE), like the SDK. If you use the model in a product, we would appreciate a "Powered by newmean.ai" mention in your docs or an about or licenses screen, and "Based on newmean-pii-medium" in the card of a model you derive from it (requests, not license conditions).

Built on mMiniLMv2 (Microsoft, MIT License) and the XLM-R SentencePiece tokenizer (Facebook AI, MIT License); see NOTICE. Made by newmean.ai — questions and commercial inquiries: hoonjae@newmean.ai.