Propella-bekko-a25m

propella-1 is a family of document annotators from ellamind. Given a document, a propella-1 model returns 18 properties of it — how complete and readable it is, how dense, educational or safe, what kind of document it is, which business sector and region it belongs to — as JSON, without a task-specific prompt.

Propella-bekko-a25m distils the largest of them, propella-1-4b, into a model small enough to run over whole corpora: the same 17 properties, predicted in one forward pass, without generating text. It has 123M parameters and annotates 188 documents per second on one RTX A6000.

Agreement with two annotators that are not the teacher — DeepSeek-V4.1-Flash on the left, GLM-5.3-Flash on the right — against throughput. Both were asked to annotate the same 300 held-out documents with propella's own prompt and schema.

What it predicts

Propella-bekko-a25m predicts 17 of propella-1's 18 properties — everything except the free-text one_sentence_description — with one head per property. The label names are propella-1's own, so an annotation from this model and one from propella-1 can be read side by side without a mapping.

country_relevance is open vocabulary: propella-1 accepts any country. In our 845k training documents 749 different values occur, of which the 64 most frequent carry 96% of the annotated countries, so this model predicts those 64 plus other — 65 values. A country outside the 64 comes back as other, which is how the property narrows in practice — the numbers behind that cut are in docs/coverage.md.

What each level means — and the rubric the annotators were given — is in the property reference: propella-1's own definitions, reproduced here so they stay online with the model.

Architecture

• Backbone: bekko-embedding-v1-a25m
• Heads: one per property, softmax for single-select and sigmoid for multi-select, with no cross-property conditioning — the properties are predicted independently from one shared document state.
• Input: 1,024 tokens of the document head plus 128 tokens of its tail, so the end of a long document is not lost; document-level length features are passed to the heads.
• Decoding: ordinal properties are decoded to the expected level (round(Σ p_i · rank_i)) rather than by argmax, which is worth about +0.013 QWK on its own.

Training

Propella-bekko-a25m was trained on propella-1-4b labels derived from the Dynaword corpora, FineWeb-2 and FinePDFs: 844,912 documents in 17 languages — Danish, Norwegian, Swedish, Icelandic, Faroese, Dutch, German, English, French, Spanish, Italian, Portuguese, Romanian, Polish, Czech, Hungarian and Finnish. Training took 316 minutes on one Quadro RTX 8000 (3 epochs, 65,536 tokens per batch), with a cross-entropy objective for the single-select properties, binary cross-entropy for the multi-select ones and a token-level F1 loss for PII. The annotations are published in propella-bekko-data and the training code in propella-bekko.

Results

As a held-out test, we measure agreement with DeepSeek-V4.1-Flash and GLM-5.3-Flash annotations on the same 300 held-out documents and report macro-F1 across all properties. Throughput is documents per second on one RTX A6000, bf16 for every model including the propella-1 baselines; chance — always predicting the most frequent label — is 0.123.

Where that single number comes from, property by property: this model's agreement with each reference annotator on the same 300 documents (DeepSeek-V4.1-Flash on the left, GLM-5.3-Flash on the right), next to propella-1-4b's own on those documents.

Against DeepSeek-V4.1-Flash the distance to the teacher sits mostly in three properties — content_length (−0.26), country_relevance (−0.23), commercial_bias (−0.16) — with a mean gap of +0.05; 6 of 17 properties are at or above propella-1-4b's own score (audience_level, content_quality, content_ratio, educational_value, information_density, pii_presence). Against GLM-5.3-Flash it is closer: mean gap +0.03, 7 of 17 at or above.

content_safety sits at the no-signal floor for both models here: on these documents it is almost always "safe", so the property says little in this comparison.

Usage

from transformers import AutoModel, AutoTokenizer

repo = "danish-foundation-models/propella-bekko-a25m"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(repo, trust_remote_code=True).eval()

print(model.annotate(tokenizer, ["a document"]))

annotate takes a document or a list of them and returns propella-style dicts, one per document: single-select properties as their label name, multi-select ones as a list of label names. trust_remote_code=True runs modeling_propella_bekko.py, the modeling file shipped in this repository — the same code as in the training repository, where the model is also importable as propella_bekko.PropellaBekko with a batched Annotator for whole corpora.

On the full document — a Dutch government report on making neighbourhoods gas-free — the model answers:

{
    "content_integrity": "complete",
    "content_ratio": "complete_content",
    "content_length": "substantial",
    "content_type": [
        "analytical",
        "technical_documentation"
    ],
    "business_sector": [
        "government_public",
        "energy_utilities",
        "environmental_services"
    ],
    "technical_content": [
        "scientific",
        "data_heavy"
    ],
    "content_quality": "excellent",
    "information_density": "dense",
    "educational_value": "moderate",
    "reasoning_indicators": "analytical",
    "audience_level": "expert",
    "commercial_bias": "none",
    "time_sensitivity": "slowly_changing",
    "content_safety": "safe",
    "pii_presence": "no_pii",
    "regional_relevance": [
        "european"
    ],
    "country_relevance": [
        "netherlands"
    ]
}

Multi-select properties are decoded at a sigmoid threshold of 0.5. thresholds.json in this repository holds thresholds tuned for macro-F1 if you would rather have rare-label recall.

For a whole corpus, the batched equivalent is propella_bekko.Annotator in the training package — the same forward pass over length-bucketed batches, several times the throughput of one document at a time.

Citation

The model and the code are MIT. If you use this model, please cite the Dynaword paper (arXiv:2508.02271) and the project:

@inproceedings{dynaword,
	title = {Dynaword: {From} {One}-shot to {Continuously} {Developed} {Datasets}},
	booktitle = {Proceedings of the {Language} {Resources} and {Evaluation} {Conference} ({LREC} 2026)},
	author = {Enevoldsen, Kenneth and Jensen, Kristian Nørgaard and Kostkan, Jan and Szabó, Balázs and Kardos, Márton and Vad, Kirsten and Núñez, Andrea Blasi and Barmina, Gianluca and Nielsen, Jacob and Larsen, Rasmus and Vahlstrup, Peter and Dalum, Per Møldrup and Elliott, Desmond and Galke, Lukas and Schneider-Kamp, Peter and Nielbo, Kristoffer},
	year = {2026},
}
@misc{enevoldsenDanishFoundationModels2023,
	title = {Danish {Foundation} {Models}},
	url = {http://arxiv.org/abs/2311.07264},
	publisher = {arXiv},
	author = {Enevoldsen, Kenneth and Hansen, Lasse and Nielsen, Dan S. and Egebæk, Rasmus A. F. and Holm, Søren V. and Nielsen, Martin C. and Bernstorff, Martin and Larsen, Rasmus and Jørgensen, Peter B. and Højmark-Bertelsen, Malte and Vahlstrup, Peter B. and Møldrup-Dalum, Per and Nielbo, Kristoffer},
	year = {2023},
}

---

A Danish Foundation Models model