longformer-base-4096-bne-es-finetuned-augmented1

This model is a fine-tuned version of PlanTL-GOB-ES/longformer-base-4096-bne-es on the None dataset. It achieves the following results on the evaluation set:

Loss: 1.7936
Precision: 0.5307
Recall: 0.6189
F1: 0.5714
Accuracy: 0.8447

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
0.723	1.0	713	0.5777	0.3428	0.5358	0.4181	0.8332
0.3414	2.0	1426	0.7079	0.3337	0.5462	0.4143	0.8210
0.1307	3.0	2139	0.7862	0.4226	0.5868	0.4913	0.8298
0.0792	4.0	2852	0.9581	0.4215	0.5906	0.4919	0.8245
0.0427	5.0	3565	1.0090	0.4451	0.6047	0.5128	0.8303
0.032	6.0	4278	1.0855	0.4865	0.6123	0.5422	0.8450
0.0237	7.0	4991	1.1150	0.4693	0.6066	0.5292	0.8455
0.0171	8.0	5704	1.1544	0.4778	0.5991	0.5316	0.8456
0.0155	9.0	6417	1.1691	0.4812	0.6038	0.5356	0.8421
0.0114	10.0	7130	1.2833	0.4861	0.6104	0.5412	0.8349
0.0137	11.0	7843	1.2716	0.4594	0.6028	0.5214	0.8334
0.0104	12.0	8556	1.2635	0.4571	0.5981	0.5182	0.8459
0.0053	13.0	9269	1.2427	0.4447	0.6066	0.5132	0.8419
0.0067	14.0	9982	1.2834	0.4862	0.6	0.5372	0.8432
0.0068	15.0	10695	1.3774	0.5012	0.6094	0.5500	0.8373
0.0077	16.0	11408	1.3625	0.4871	0.6057	0.5399	0.8428
0.0051	17.0	12121	1.3764	0.5	0.6113	0.5501	0.8445
0.0061	18.0	12834	1.5528	0.4613	0.6009	0.5219	0.8267
0.0049	19.0	13547	1.3307	0.5070	0.6151	0.5558	0.8538
0.0059	20.0	14260	1.3556	0.4903	0.6198	0.5475	0.8439
0.0064	21.0	14973	1.4599	0.5004	0.6123	0.5507	0.8409
0.0057	22.0	15686	1.3506	0.5077	0.6217	0.5589	0.8439
0.0054	23.0	16399	1.5439	0.4914	0.5953	0.5384	0.8377
0.0034	24.0	17112	1.5174	0.5059	0.6066	0.5517	0.8377
0.0048	25.0	17825	1.5228	0.4984	0.6057	0.5468	0.8438
0.0041	26.0	18538	1.4479	0.5224	0.6057	0.5609	0.8403
0.0049	27.0	19251	1.3992	0.5291	0.6349	0.5772	0.8447
0.0048	28.0	19964	1.4971	0.5234	0.6321	0.5726	0.8478
0.0018	29.0	20677	1.4874	0.4981	0.6151	0.5504	0.8390
0.0035	30.0	21390	1.3051	0.5051	0.6094	0.5524	0.8421
0.0031	31.0	22103	1.5998	0.5133	0.6179	0.5608	0.8364
0.0031	32.0	22816	1.4274	0.5085	0.6179	0.5579	0.8458
0.0042	33.0	23529	1.3180	0.5111	0.6066	0.5548	0.8494
0.0022	34.0	24242	1.5043	0.4886	0.6085	0.5420	0.8442
0.0021	35.0	24955	1.5247	0.4962	0.6094	0.5470	0.8425
0.0024	36.0	25668	1.5139	0.4851	0.5981	0.5357	0.8432
0.0027	37.0	26381	1.5214	0.4930	0.6009	0.5417	0.8404
0.0024	38.0	27094	1.4470	0.5087	0.6075	0.5537	0.8472
0.0009	39.0	27807	1.4867	0.5016	0.6094	0.5503	0.8485
0.0015	40.0	28520	1.5234	0.5148	0.6217	0.5632	0.8483
0.0023	41.0	29233	1.5742	0.4926	0.6264	0.5515	0.8407
0.0017	42.0	29946	1.5897	0.5252	0.6	0.5601	0.8403
0.0022	43.0	30659	1.4243	0.4889	0.6038	0.5403	0.8448
0.001	44.0	31372	1.6117	0.5081	0.6179	0.5577	0.8462
0.0015	45.0	32085	1.5342	0.5169	0.6066	0.5582	0.8405
0.0005	46.0	32798	1.5110	0.4687	0.6142	0.5316	0.8432
0.0019	47.0	33511	1.5835	0.5066	0.6132	0.5548	0.8427
0.0063	48.0	34224	1.5688	0.5058	0.5802	0.5404	0.8394
0.0017	49.0	34937	1.5410	0.5075	0.6028	0.5511	0.8419
0.0012	50.0	35650	1.5343	0.5220	0.5943	0.5558	0.8359
0.0009	51.0	36363	1.5190	0.5173	0.6358	0.5705	0.8411
0.0006	52.0	37076	1.6576	0.5066	0.6189	0.5571	0.8311
0.0009	53.0	37789	1.5675	0.5155	0.6283	0.5663	0.8475
0.0007	54.0	38502	1.6993	0.5218	0.6208	0.5670	0.8328
0.0019	55.0	39215	1.6003	0.5284	0.6047	0.5640	0.8365
0.0014	56.0	39928	1.4922	0.5428	0.6226	0.5800	0.8556
0.0004	57.0	40641	1.5974	0.5402	0.6142	0.5748	0.8464
0.0002	58.0	41354	1.7351	0.5501	0.6113	0.5791	0.8417
0.0008	59.0	42067	1.6191	0.5179	0.6132	0.5616	0.8476
0.0006	60.0	42780	1.5721	0.5059	0.6094	0.5528	0.8455
0.0009	61.0	43493	1.6079	0.4980	0.6	0.5443	0.8388
0.0011	62.0	44206	1.7208	0.4907	0.5943	0.5375	0.8288
0.0002	63.0	44919	1.7335	0.5012	0.5925	0.5430	0.8354
0.001	64.0	45632	1.7670	0.5439	0.6189	0.5790	0.8352
0.0002	65.0	46345	1.7687	0.5203	0.6170	0.5645	0.8430
0.0002	66.0	47058	1.7857	0.5059	0.6066	0.5517	0.8375
0.0003	67.0	47771	1.7961	0.5090	0.6104	0.5551	0.8335
0.0009	68.0	48484	1.7072	0.5039	0.6132	0.5532	0.8416
0.0003	69.0	49197	1.7345	0.5147	0.6113	0.5589	0.8421
0.0002	70.0	49910	1.6423	0.5427	0.6179	0.5779	0.8491
0.0007	71.0	50623	1.6966	0.5422	0.6368	0.5857	0.8425
0.0016	72.0	51336	1.7376	0.5153	0.6198	0.5627	0.8349
0.001	73.0	52049	1.6447	0.51	0.6255	0.5619	0.8442
0.0001	74.0	52762	1.7449	0.5204	0.6132	0.5630	0.8421
0.0002	75.0	53475	1.6948	0.5287	0.6179	0.5698	0.8450
0.0005	76.0	54188	1.6546	0.5305	0.6321	0.5768	0.8480
0.0002	77.0	54901	1.7188	0.5224	0.6264	0.5697	0.8444
0.0001	78.0	55614	1.6167	0.5102	0.6142	0.5574	0.8462
0.0005	79.0	56327	1.6709	0.5160	0.6245	0.5651	0.8462
0.0	80.0	57040	1.6883	0.5223	0.6179	0.5661	0.8475
0.0002	81.0	57753	1.7612	0.5051	0.6057	0.5508	0.8436
0.0001	82.0	58466	1.7766	0.5342	0.6189	0.5734	0.8410
0.0001	83.0	59179	1.7235	0.5252	0.6189	0.5682	0.8453
0.0002	84.0	59892	1.7663	0.5319	0.6208	0.5729	0.8440
0.0007	85.0	60605	1.7581	0.5280	0.6217	0.5711	0.8408
0.0002	86.0	61318	1.7467	0.5271	0.6236	0.5713	0.8407
0.0003	87.0	62031	1.7220	0.5275	0.6151	0.5679	0.8437
0.0001	88.0	62744	1.7616	0.5207	0.6179	0.5651	0.8430
0.0	89.0	63457	1.7817	0.5396	0.6170	0.5757	0.8460
0.0	90.0	64170	1.7684	0.5319	0.6132	0.5697	0.8436
0.0	91.0	64883	1.7731	0.5264	0.6208	0.5697	0.8434
0.0	92.0	65596	1.7448	0.5314	0.6236	0.5738	0.8467
0.0	93.0	66309	1.7457	0.5353	0.6302	0.5789	0.8484
0.0	94.0	67022	1.7477	0.5424	0.6274	0.5818	0.8485
0.0	95.0	67735	1.7931	0.5292	0.6160	0.5693	0.8444
0.0002	96.0	68448	1.8056	0.5287	0.6170	0.5694	0.8455
0.0001	97.0	69161	1.7963	0.5247	0.6217	0.5691	0.8450
0.0001	98.0	69874	1.7963	0.5211	0.6179	0.5654	0.8446
0.0001	99.0	70587	1.7950	0.5261	0.6189	0.5687	0.8452
0.0002	100.0	71300	1.7936	0.5307	0.6189	0.5714	0.8447

Framework versions

Transformers 4.28.1
Pytorch 2.0.0+cu117
Datasets 2.12.0
Tokenizers 0.13.3

joheras/longformer-base-4096-bne-es-finetuned-augmented1

longformer-base-4096-bne-es-finetuned-augmented1

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

Seed this model