clinico-xlm-roberta

This model is a fine-tuned version of xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

Loss: 1.2239
Precision: 0.4333
Recall: 0.5984
F1: 0.5026
Accuracy: 0.8446

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 32
eval_batch_size: 32
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	1.0	25	1.4128	0.0015	0.0023	0.0018	0.5371
No log	2.0	50	1.0675	0.0260	0.0595	0.0362	0.6283
No log	3.0	75	0.9345	0.0456	0.1213	0.0663	0.6667
No log	4.0	100	0.8709	0.0707	0.1465	0.0954	0.7060
No log	5.0	125	0.8154	0.1112	0.2059	0.1444	0.7399
No log	6.0	150	0.8001	0.1615	0.3066	0.2116	0.7347
No log	7.0	175	0.6928	0.2129	0.3616	0.2680	0.7846
No log	8.0	200	0.6576	0.2210	0.3753	0.2782	0.7988
No log	9.0	225	0.6174	0.2263	0.4119	0.2921	0.8120
No log	10.0	250	0.6232	0.2385	0.4268	0.3060	0.8150
No log	11.0	275	0.6304	0.2528	0.4577	0.3257	0.8237
No log	12.0	300	0.6562	0.2758	0.4714	0.3480	0.8204
No log	13.0	325	0.6725	0.2726	0.4828	0.3485	0.8164
No log	14.0	350	0.6959	0.2732	0.4943	0.3519	0.8216
No log	15.0	375	0.6838	0.2919	0.5046	0.3698	0.8313
No log	16.0	400	0.7033	0.3199	0.5252	0.3976	0.8307
No log	17.0	425	0.7198	0.2848	0.4897	0.3601	0.8094
No log	18.0	450	0.7319	0.3070	0.5149	0.3846	0.8293
No log	19.0	475	0.7841	0.3184	0.5275	0.3971	0.8283
0.5744	20.0	500	0.8119	0.2995	0.5229	0.3808	0.8204
0.5744	21.0	525	0.7665	0.2914	0.5069	0.3701	0.8228
0.5744	22.0	550	0.8008	0.3062	0.5172	0.3847	0.8201
0.5744	23.0	575	0.7822	0.3008	0.5217	0.3816	0.8294
0.5744	24.0	600	0.8432	0.3148	0.5114	0.3897	0.8191
0.5744	25.0	625	0.8161	0.3387	0.5309	0.4135	0.8332
0.5744	26.0	650	0.8405	0.3289	0.5423	0.4095	0.8275
0.5744	27.0	675	0.8273	0.3465	0.5435	0.4232	0.8311
0.5744	28.0	700	0.8920	0.3326	0.5446	0.4130	0.8309
0.5744	29.0	725	0.8796	0.3303	0.5400	0.4099	0.8344
0.5744	30.0	750	0.8918	0.3319	0.5229	0.4060	0.8246
0.5744	31.0	775	0.8656	0.3613	0.5618	0.4398	0.8381
0.5744	32.0	800	0.9315	0.3375	0.5503	0.4184	0.8260
0.5744	33.0	825	0.9042	0.3644	0.5686	0.4441	0.8339
0.5744	34.0	850	0.9060	0.3865	0.5652	0.4591	0.8387
0.5744	35.0	875	0.9413	0.4021	0.5778	0.4742	0.8360
0.5744	36.0	900	0.9608	0.3634	0.5629	0.4417	0.8337
0.5744	37.0	925	0.8908	0.3536	0.5526	0.4313	0.8355
0.5744	38.0	950	0.9339	0.3543	0.5744	0.4382	0.8360
0.5744	39.0	975	0.9853	0.3751	0.5721	0.4531	0.8416
0.068	40.0	1000	0.9807	0.4005	0.5847	0.4753	0.8352
0.068	41.0	1025	1.0515	0.3953	0.5641	0.4649	0.8290
0.068	42.0	1050	0.9588	0.3912	0.5778	0.4665	0.8400
0.068	43.0	1075	0.9839	0.3888	0.5858	0.4674	0.8381
0.068	44.0	1100	1.0556	0.4092	0.5721	0.4771	0.8341
0.068	45.0	1125	0.9591	0.4097	0.5892	0.4833	0.8433
0.068	46.0	1150	1.0339	0.4057	0.5904	0.4809	0.8337
0.068	47.0	1175	1.0162	0.3871	0.5904	0.4676	0.8438
0.068	48.0	1200	1.0642	0.3864	0.5858	0.4657	0.8348
0.068	49.0	1225	1.0270	0.4257	0.5904	0.4947	0.8464
0.068	50.0	1250	1.0872	0.4126	0.6053	0.4907	0.8390
0.068	51.0	1275	1.0346	0.4086	0.5904	0.4829	0.8437
0.068	52.0	1300	1.0785	0.4131	0.6007	0.4895	0.8389
0.068	53.0	1325	1.0533	0.4380	0.5984	0.5058	0.8433
0.068	54.0	1350	1.0574	0.4109	0.5961	0.4865	0.8430
0.068	55.0	1375	1.1087	0.4166	0.5973	0.4908	0.8417
0.068	56.0	1400	1.0861	0.4140	0.5870	0.4856	0.8398
0.068	57.0	1425	1.0796	0.4085	0.6053	0.4878	0.8442
0.068	58.0	1450	1.1179	0.4208	0.6053	0.4965	0.8383
0.068	59.0	1475	1.1096	0.3950	0.5915	0.4737	0.8416
0.0173	60.0	1500	1.0741	0.4518	0.6167	0.5215	0.8440
0.0173	61.0	1525	1.0957	0.4536	0.6098	0.5203	0.8423
0.0173	62.0	1550	1.1131	0.4581	0.5881	0.5150	0.8455
0.0173	63.0	1575	1.0809	0.4367	0.6156	0.5109	0.8499
0.0173	64.0	1600	1.1138	0.4439	0.5927	0.5076	0.8419
0.0173	65.0	1625	1.1543	0.4100	0.5995	0.4870	0.8394
0.0173	66.0	1650	1.1292	0.4256	0.6087	0.5009	0.8432
0.0173	67.0	1675	1.1415	0.4542	0.6064	0.5194	0.8461
0.0173	68.0	1700	1.1804	0.4300	0.6007	0.5012	0.8436
0.0173	69.0	1725	1.1676	0.4356	0.5995	0.5046	0.8437
0.0173	70.0	1750	1.1806	0.4316	0.5961	0.5007	0.8420
0.0173	71.0	1775	1.1530	0.435	0.5973	0.5034	0.8459
0.0173	72.0	1800	1.1691	0.4344	0.5984	0.5034	0.8435
0.0173	73.0	1825	1.1869	0.4242	0.5927	0.4945	0.8410
0.0173	74.0	1850	1.1868	0.4450	0.5927	0.5083	0.8395
0.0173	75.0	1875	1.1987	0.4458	0.6064	0.5138	0.8398
0.0173	76.0	1900	1.1936	0.4396	0.5870	0.5027	0.8392
0.0173	77.0	1925	1.1882	0.4433	0.5950	0.5081	0.8414
0.0173	78.0	1950	1.2038	0.4387	0.5938	0.5046	0.8413
0.0173	79.0	1975	1.2103	0.4417	0.5984	0.5083	0.8403
0.0056	80.0	2000	1.2062	0.4259	0.5915	0.4952	0.8394
0.0056	81.0	2025	1.1871	0.4536	0.5984	0.5160	0.8425
0.0056	82.0	2050	1.1944	0.4268	0.6007	0.4990	0.8416
0.0056	83.0	2075	1.1941	0.4549	0.6007	0.5178	0.8447
0.0056	84.0	2100	1.2032	0.4553	0.6007	0.5180	0.8436
0.0056	85.0	2125	1.2096	0.4420	0.6018	0.5097	0.8414
0.0056	86.0	2150	1.2011	0.4333	0.6018	0.5038	0.8401
0.0056	87.0	2175	1.2329	0.4511	0.5961	0.5136	0.8411
0.0056	88.0	2200	1.2134	0.4523	0.6018	0.5164	0.8429
0.0056	89.0	2225	1.2281	0.4410	0.5984	0.5078	0.8426
0.0056	90.0	2250	1.2284	0.4490	0.6041	0.5151	0.8416
0.0056	91.0	2275	1.2129	0.435	0.5973	0.5034	0.8438
0.0056	92.0	2300	1.2164	0.4387	0.5973	0.5058	0.8428
0.0056	93.0	2325	1.2177	0.4429	0.6030	0.5107	0.8433
0.0056	94.0	2350	1.2297	0.4545	0.6053	0.5191	0.8434
0.0056	95.0	2375	1.2243	0.4579	0.6030	0.5205	0.8459
0.0056	96.0	2400	1.2241	0.4478	0.6041	0.5144	0.8457
0.0056	97.0	2425	1.2286	0.4496	0.6018	0.5147	0.8434
0.0056	98.0	2450	1.2279	0.4426	0.5995	0.5092	0.8431
0.0056	99.0	2475	1.2244	0.4328	0.6007	0.5031	0.8443
0.0029	100.0	2500	1.2239	0.4333	0.5984	0.5026	0.8446

Framework versions

Transformers 4.27.0.dev0
Pytorch 1.13.0
Datasets 2.8.0
Tokenizers 0.12.1

joheras/clinico-xlm-roberta

clinico-xlm-roberta

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

为此模型做种