Historisches Grundbuch der Stadt Basel Nested NER
Nested tagger for Historical Texts, mainly 15th to 18th century German.
Developed by Ismail Prada Ziegler.
A model for historical German developed as part of the project Economies of Space. Practices, Discourses, and Actors on the Basel Real Estate Market (1400-1700) at the University of Basel and the Digital Humanities Bern. This Model was created to annotate nested document structures. It can be used to annotate flat text (such as in the example), but may perform slightly worse than models trained only for that task. You can annotate nested tags by using this script. You can find more info on this strategy here.
Performance
This model features a total of 47 classes. Each class except pro (pronouns) consists of an element and attribute part.
The scores here do represent the evaluation by the Flair model trainer. Please note that actual scores for nested elements may be lower when elements around them were not identified successfully.
| Label | Precision | Recall | F1-Score | Support |
|---|---|---|---|---|
| head.nam | 91.08% | 93.33% | 92.19% | 525 |
| reference.per | 82.41% | 86.77% | 84.54% | 378 |
| reference.fac | 70.10% | 76.17% | 73.01% | 277 |
| value.date | 91.51% | 94.80% | 93.12% | 250 |
| head.type | 84.83% | 81.62% | 83.20% | 185 |
| value.money | 84.18% | 81.42% | 82.78% | 183 |
| head.occ | 86.11% | 87.74% | 86.92% | 106 |
| attr.loc | 71.82% | 79.80% | 75.60% | 99 |
| trigger.due | 77.45% | 74.53% | 75.96% | 106 |
| pro | 88.89% | 87.13% | 88.00% | 101 |
| appo.occ | 84.38% | 85.26% | 84.82% | 95 |
| attr.owner | 91.18% | 82.67% | 86.71% | 75 |
| reference.org | 92.16% | 77.05% | 83.93% | 61 |
| head.fam | 89.13% | 93.18% | 91.11% | 44 |
| appo.fam | 92.68% | 92.68% | 92.68% | 41 |
| other.other | 63.64% | 43.75% | 51.85% | 48 |
| appo.nam | 75.61% | 77.50% | 76.54% | 40 |
| reference.loc | 71.88% | 63.89% | 67.65% | 36 |
| reference.gpe-loc | 93.55% | 82.86% | 87.88% | 35 |
| value.time | 87.50% | 87.50% | 87.50% | 32 |
| trigger.other | 36.36% | 47.06% | 41.03% | 17 |
| trigger.sale | 93.33% | 82.35% | 87.50% | 17 |
| head.title | 87.50% | 87.50% | 87.50% | 16 |
| attr.includes | 78.57% | 64.71% | 70.97% | 17 |
| appo.title | 75.00% | 56.25% | 64.29% | 16 |
| attr.dead | 92.31% | 92.31% | 92.31% | 13 |
| trigger.rent | 63.64% | 58.33% | 60.87% | 12 |
| reference.gpe-org | 76.92% | 100.00% | 86.96% | 10 |
| attr.dues | 75.00% | 81.82% | 78.26% | 11 |
| appo.org-aff | 60.00% | 54.55% | 57.14% | 11 |
| trigger.seizure | 100.00% | 72.73% | 84.21% | 11 |
| head.org-aff | 100.00% | 90.00% | 94.74% | 10 |
| head.role | 71.43% | 62.50% | 66.67% | 8 |
| appo.role | 100.00% | 8.33% | 15.38% | 12 |
| head.other | 50.00% | 60.00% | 54.55% | 5 |
| head.heir | 100.00% | 100.00% | 100.00% | 5 |
| appo.gen | 100.00% | 100.00% | 100.00% | 4 |
| head.gen | 75.00% | 75.00% | 75.00% | 4 |
| trigger.redemption | 60.00% | 100.00% | 75.00% | 3 |
| attr.other | 100.00% | 25.00% | 40.00% | 4 |
| attr.org-aff | 100.00% | 66.67% | 80.00% | 3 |
| trigger.litigation | 50.00% | 50.00% | 50.00% | 2 |
| reference.other | 0.00% | 0.00% | 0.00% | 4 |
| trigger.testament | 0.00% | 0.00% | 0.00% | 3 |
| appo.other | 0.00% | 0.00% | 0.00% | 2 |
| head.owner | 100.00% | 100.00% | 100.00% | 1 |
| unclear.other | 0.00% | 0.00% | 0.00% | 1 |
Dataset
Manually annotated dataset created from the Historical Land Registry of the city of Basel. Timeframe: 1400-1700. Language: Early New High German. Language models based on the full HLRB corpus until 1800, appr. 120k documents.
The documents were annotated according to the BeNASch annotation guidelines. For this model, a simplified tagset was used.
The training data was prepared in a special way to accommodate nested annotation. See the linked paper for more information. In addition to the strategy described in the paper, for this version two augmentations were applied:
Class of Annotation Layer
Each input sentence was prefixed and suffixed with a marker informing the prediction which surrounding element is present.
Context information
In addition, the input is prefixed and suffixed with the previous and following 5 tokens.
Example input
When predicting the nested elements inside "Die Lütkilchen s . Ulrich", the input should look like this
[B-CONTEXT] [B-REFERENCE-ORG] Die Lütkilchen s . Ulrich [E-REFERENCE-ORG] hat Caspar Vogt des Küeffers [E-CONTEXT]
Acknowledgments
Calculations were performed on UBELIX (https://www.id.unibe.ch/hpc), the HPC cluster at the University of Bern.
Citation
If you publish works using this model, please cite:
Prada Ziegler, I. (2024, May 30). What's in an entity? Exploring Nested Named Entity Recognition in the Historical Land Register of Basel (1400-1700). DH Benelux 2024, Leuven, Belgium. Zenodo. https://doi.org/10.5281/zenodo.11394453