layoutlm-funsd
This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:
- Loss: 1.1017
- Answer: {'precision': 0.40439158279963405, 'recall': 0.546353522867738, 'f1': 0.46477392218717145, 'number': 809}
- Header: {'precision': 0.3368421052631579, 'recall': 0.2689075630252101, 'f1': 0.29906542056074764, 'number': 119}
- Question: {'precision': 0.5619128949615713, 'recall': 0.6178403755868545, 'f1': 0.5885509838998211, 'number': 1065}
- Overall Precision: 0.4799
- Overall Recall: 0.5680
- Overall F1: 0.5202
- Overall Accuracy: 0.6339
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 32
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|
| 1.3745 | 1.0 | 5 | 1.1446 | {'precision': 0.24631396357328708, 'recall': 0.3510506798516687, 'f1': 0.28950050968399593, 'number': 809} | {'precision': 0.20930232558139536, 'recall': 0.226890756302521, 'f1': 0.21774193548387097, 'number': 119} | {'precision': 0.4135151890886547, 'recall': 0.6262910798122066, 'f1': 0.4981329350261389, 'number': 1065} | 0.3378 | 0.4907 | 0.4002 | 0.5475 |
| 1.0195 | 2.0 | 10 | 1.0518 | {'precision': 0.29006968641114983, 'recall': 0.411619283065513, 'f1': 0.340316811446091, 'number': 809} | {'precision': 0.2815533980582524, 'recall': 0.24369747899159663, 'f1': 0.26126126126126126, 'number': 119} | {'precision': 0.42618741976893454, 'recall': 0.6234741784037559, 'f1': 0.5062905070529927, 'number': 1065} | 0.3653 | 0.5148 | 0.4273 | 0.5967 |
| 0.8996 | 3.0 | 15 | 1.0952 | {'precision': 0.3147887323943662, 'recall': 0.5525339925834364, 'f1': 0.4010767160161508, 'number': 809} | {'precision': 0.25301204819277107, 'recall': 0.17647058823529413, 'f1': 0.20792079207920794, 'number': 119} | {'precision': 0.4714285714285714, 'recall': 0.5267605633802817, 'f1': 0.4975609756097561, 'number': 1065} | 0.3821 | 0.5163 | 0.4392 | 0.5831 |
| 0.8294 | 4.0 | 20 | 1.0418 | {'precision': 0.3429571303587052, 'recall': 0.484548825710754, 'f1': 0.4016393442622951, 'number': 809} | {'precision': 0.32, 'recall': 0.20168067226890757, 'f1': 0.24742268041237112, 'number': 119} | {'precision': 0.49588815789473684, 'recall': 0.5661971830985916, 'f1': 0.5287154756685665, 'number': 1065} | 0.4187 | 0.5113 | 0.4604 | 0.6110 |
| 0.773 | 5.0 | 25 | 1.0412 | {'precision': 0.34150772025431425, 'recall': 0.4647713226205192, 'f1': 0.393717277486911, 'number': 809} | {'precision': 0.2857142857142857, 'recall': 0.20168067226890757, 'f1': 0.23645320197044337, 'number': 119} | {'precision': 0.4541223404255319, 'recall': 0.6413145539906103, 'f1': 0.5317244063838069, 'number': 1065} | 0.4028 | 0.5434 | 0.4626 | 0.6114 |
| 0.731 | 6.0 | 30 | 1.0832 | {'precision': 0.352991452991453, 'recall': 0.5105067985166872, 'f1': 0.4173825164224356, 'number': 809} | {'precision': 0.2708333333333333, 'recall': 0.2184873949579832, 'f1': 0.24186046511627907, 'number': 119} | {'precision': 0.5029686174724343, 'recall': 0.5568075117370892, 'f1': 0.5285204991087344, 'number': 1065} | 0.4221 | 0.5178 | 0.4651 | 0.6014 |
| 0.6884 | 7.0 | 35 | 1.1304 | {'precision': 0.3588709677419355, 'recall': 0.5500618046971569, 'f1': 0.4343582235236701, 'number': 809} | {'precision': 0.36619718309859156, 'recall': 0.2184873949579832, 'f1': 0.2736842105263158, 'number': 119} | {'precision': 0.5510204081632653, 'recall': 0.5577464788732395, 'f1': 0.5543630424638357, 'number': 1065} | 0.4458 | 0.5344 | 0.4861 | 0.6078 |
| 0.6731 | 8.0 | 40 | 1.0667 | {'precision': 0.3651096282173499, 'recall': 0.47342398022249693, 'f1': 0.41227125941872983, 'number': 809} | {'precision': 0.29, 'recall': 0.24369747899159663, 'f1': 0.2648401826484018, 'number': 119} | {'precision': 0.49964912280701756, 'recall': 0.6685446009389672, 'f1': 0.5718875502008032, 'number': 1065} | 0.4367 | 0.5640 | 0.4922 | 0.6205 |
| 0.6441 | 9.0 | 45 | 1.0893 | {'precision': 0.3948576675849403, 'recall': 0.5315203955500618, 'f1': 0.45310853530031614, 'number': 809} | {'precision': 0.3238095238095238, 'recall': 0.2857142857142857, 'f1': 0.30357142857142855, 'number': 119} | {'precision': 0.5439367311072056, 'recall': 0.5812206572769953, 'f1': 0.5619609623241035, 'number': 1065} | 0.4644 | 0.5434 | 0.5008 | 0.6241 |
| 0.6139 | 10.0 | 50 | 1.0987 | {'precision': 0.37037037037037035, 'recall': 0.5562422744128553, 'f1': 0.44466403162055335, 'number': 809} | {'precision': 0.313953488372093, 'recall': 0.226890756302521, 'f1': 0.2634146341463415, 'number': 119} | {'precision': 0.533678756476684, 'recall': 0.5802816901408451, 'f1': 0.5560053981106613, 'number': 1065} | 0.4453 | 0.5494 | 0.4919 | 0.6253 |
| 0.6007 | 11.0 | 55 | 1.0803 | {'precision': 0.40096618357487923, 'recall': 0.5129789864029666, 'f1': 0.45010845986984815, 'number': 809} | {'precision': 0.29591836734693877, 'recall': 0.24369747899159663, 'f1': 0.26728110599078336, 'number': 119} | {'precision': 0.5409054805401112, 'recall': 0.6394366197183099, 'f1': 0.5860585197934596, 'number': 1065} | 0.4703 | 0.5645 | 0.5131 | 0.6317 |
| 0.5985 | 12.0 | 60 | 1.0997 | {'precision': 0.4080846968238691, 'recall': 0.5241038318912238, 'f1': 0.45887445887445893, 'number': 809} | {'precision': 0.31683168316831684, 'recall': 0.2689075630252101, 'f1': 0.29090909090909095, 'number': 119} | {'precision': 0.5536303630363036, 'recall': 0.6300469483568075, 'f1': 0.5893719806763285, 'number': 1065} | 0.4792 | 0.5655 | 0.5188 | 0.6323 |
| 0.5828 | 13.0 | 65 | 1.0996 | {'precision': 0.40275229357798165, 'recall': 0.5426452410383189, 'f1': 0.46234860452869925, 'number': 809} | {'precision': 0.33695652173913043, 'recall': 0.2605042016806723, 'f1': 0.29383886255924174, 'number': 119} | {'precision': 0.5685936151855048, 'recall': 0.6187793427230047, 'f1': 0.5926258992805755, 'number': 1065} | 0.4823 | 0.5665 | 0.5210 | 0.6345 |
| 0.5656 | 14.0 | 70 | 1.1065 | {'precision': 0.40542986425339367, 'recall': 0.553770086526576, 'f1': 0.46812957157784746, 'number': 809} | {'precision': 0.32967032967032966, 'recall': 0.25210084033613445, 'f1': 0.28571428571428575, 'number': 119} | {'precision': 0.5730735163861824, 'recall': 0.6075117370892019, 'f1': 0.5897903372835004, 'number': 1065} | 0.4839 | 0.5645 | 0.5211 | 0.6338 |
| 0.5625 | 15.0 | 75 | 1.1017 | {'precision': 0.40439158279963405, 'recall': 0.546353522867738, 'f1': 0.46477392218717145, 'number': 809} | {'precision': 0.3368421052631579, 'recall': 0.2689075630252101, 'f1': 0.29906542056074764, 'number': 119} | {'precision': 0.5619128949615713, 'recall': 0.6178403755868545, 'f1': 0.5885509838998211, 'number': 1065} | 0.4799 | 0.5680 | 0.5202 | 0.6339 |
Framework versions
- Transformers 4.33.0
- Pytorch 2.0.1+cu118
- Datasets 2.14.5
- Tokenizers 0.13.3