model
This model is a fine-tuned version of sentence-transformers/paraphrase-MiniLM-L3-v2 on the nyt_ingredients dataset. It achieves the following results on the evaluation set:
- Loss: 0.4745
- Comment: {'precision': 0.6381763059701493, 'recall': 0.7527162701141521, 'f1': 0.6907301066447908, 'number': 7271}
- Name: {'precision': 0.7925138150349286, 'recall': 0.8159081150708458, 'f1': 0.8040408314380917, 'number': 9316}
- Qty: {'precision': 0.9870301746956062, 'recall': 0.9904382470119522, 'f1': 0.988731274028901, 'number': 7530}
- Range End: {'precision': 0.6532258064516129, 'recall': 0.9310344827586207, 'f1': 0.7677725118483412, 'number': 87}
- Unit: {'precision': 0.9281956050758279, 'recall': 0.9844083374364024, 'f1': 0.9554759060135404, 'number': 6093}
- Overall Precision: 0.8236
- Overall Recall: 0.8783
- Overall F1: 0.8501
- Overall Accuracy: 0.8310
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3
Training results
| Training Loss | Epoch | Step | Validation Loss | Comment | Name | Qty | Range End | Unit | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.5473 | 0.2 | 1000 | 0.5439 | {'precision': 0.53239608801956, 'recall': 0.6330862043901729, 'f1': 0.5783916594727406, 'number': 6879} | {'precision': 0.7656748140276302, 'recall': 0.816245610060043, 'f1': 0.7901518890168339, 'number': 8827} | {'precision': 0.9752864835013116, 'recall': 0.9824756606397774, 'f1': 0.9788678722372341, 'number': 7190} | {'precision': 0.6060606060606061, 'recall': 0.7317073170731707, 'f1': 0.6629834254143646, 'number': 82} | {'precision': 0.923214867949136, 'recall': 0.9828184658104825, 'f1': 0.9520847343644923, 'number': 5762} | 0.7837 | 0.8471 | 0.8142 | 0.8057 |
| 0.5634 | 0.4 | 2000 | 0.5237 | {'precision': 0.5564878997932629, 'recall': 0.6652129670010176, 'f1': 0.6060124486822938, 'number': 6879} | {'precision': 0.7951952610794208, 'recall': 0.8212303160756769, 'f1': 0.8080031209942595, 'number': 8827} | {'precision': 0.9757675891504888, 'recall': 0.9856745479833101, 'f1': 0.9806960492631287, 'number': 7190} | {'precision': 0.5725806451612904, 'recall': 0.8658536585365854, 'f1': 0.6893203883495146, 'number': 82} | {'precision': 0.9235782955841616, 'recall': 0.9836862200624783, 'f1': 0.9526850995882007, 'number': 5762} | 0.7987 | 0.8577 | 0.8272 | 0.8120 |
| 0.5535 | 0.59 | 3000 | 0.5022 | {'precision': 0.5893937596393404, 'recall': 0.7221979938944614, 'f1': 0.6490723804546643, 'number': 6879} | {'precision': 0.7913148371531966, 'recall': 0.8174917865639515, 'f1': 0.8041903488242506, 'number': 8827} | {'precision': 0.9812708102108768, 'recall': 0.9837273991655077, 'f1': 0.9824975691068204, 'number': 7190} | {'precision': 0.562962962962963, 'recall': 0.926829268292683, 'f1': 0.7004608294930875, 'number': 82} | {'precision': 0.931615460852329, 'recall': 0.9788267962513016, 'f1': 0.9546377792823292, 'number': 5762} | 0.8070 | 0.8689 | 0.8368 | 0.8213 |
| 0.5366 | 0.79 | 4000 | 0.4892 | {'precision': 0.6037854098771622, 'recall': 0.7002471289431603, 'f1': 0.6484485427744499, 'number': 6879} | {'precision': 0.7957470010905126, 'recall': 0.826668177183641, 'f1': 0.8109129299327665, 'number': 8827} | {'precision': 0.9751884852638794, 'recall': 0.9894297635605007, 'f1': 0.9822575077666552, 'number': 7190} | {'precision': 0.5652173913043478, 'recall': 0.9512195121951219, 'f1': 0.7090909090909091, 'number': 82} | {'precision': 0.9284076015727392, 'recall': 0.9835126692120791, 'f1': 0.955166020562953, 'number': 5762} | 0.8139 | 0.8689 | 0.8405 | 0.8251 |
| 0.5256 | 0.99 | 5000 | 0.4813 | {'precision': 0.6161294276259346, 'recall': 0.730774821921791, 'f1': 0.6685729485303898, 'number': 6879} | {'precision': 0.7992788461538461, 'recall': 0.8287073750991277, 'f1': 0.8137271260915513, 'number': 8827} | {'precision': 0.9784340659340659, 'recall': 0.9906815020862308, 'f1': 0.9845196959225985, 'number': 7190} | {'precision': 0.6330275229357798, 'recall': 0.8414634146341463, 'f1': 0.7225130890052357, 'number': 82} | {'precision': 0.9291687161829808, 'recall': 0.9835126692120791, 'f1': 0.9555686704325098, 'number': 5762} | 0.8182 | 0.8769 | 0.8465 | 0.8299 |
| 0.5079 | 1.19 | 6000 | 0.4766 | {'precision': 0.6228698444060262, 'recall': 0.7332461113533943, 'f1': 0.6735661347399347, 'number': 6879} | {'precision': 0.8044889426779623, 'recall': 0.82836750877988, 'f1': 0.8162536280419737, 'number': 8827} | {'precision': 0.9840742279462679, 'recall': 0.988317107093185, 'f1': 0.9861911040177642, 'number': 7190} | {'precision': 0.6306306306306306, 'recall': 0.8536585365853658, 'f1': 0.7253886010362693, 'number': 82} | {'precision': 0.928082191780822, 'recall': 0.9876778896216591, 'f1': 0.9569530855893728, 'number': 5762} | 0.8229 | 0.8776 | 0.8494 | 0.8313 |
| 0.5047 | 1.39 | 7000 | 0.4780 | {'precision': 0.6244848484848485, 'recall': 0.7489460677424045, 'f1': 0.6810760790534734, 'number': 6879} | {'precision': 0.8084753263996459, 'recall': 0.8278010649144669, 'f1': 0.8180240694094598, 'number': 8827} | {'precision': 0.9799036476256022, 'recall': 0.990125173852573, 'f1': 0.9849878934624697, 'number': 7190} | {'precision': 0.5923076923076923, 'recall': 0.9390243902439024, 'f1': 0.7264150943396225, 'number': 82} | {'precision': 0.9348113831899404, 'recall': 0.9805623047552933, 'f1': 0.9571404370658986, 'number': 5762} | 0.8235 | 0.8805 | 0.8511 | 0.8305 |
| 0.4912 | 1.58 | 8000 | 0.4725 | {'precision': 0.6316635745207174, 'recall': 0.7424044192469835, 'f1': 0.6825715049452018, 'number': 6879} | {'precision': 0.8068570168669386, 'recall': 0.8291605301914581, 'f1': 0.8178567437702537, 'number': 8827} | {'precision': 0.9846047156726768, 'recall': 0.9873435326842838, 'f1': 0.9859722222222222, 'number': 7190} | {'precision': 0.6428571428571429, 'recall': 0.8780487804878049, 'f1': 0.7422680412371134, 'number': 82} | {'precision': 0.9298820445609436, 'recall': 0.9850746268656716, 'f1': 0.9566829597168379, 'number': 5762} | 0.8264 | 0.8794 | 0.8521 | 0.8342 |
| 0.4955 | 1.78 | 9000 | 0.4725 | {'precision': 0.6421661012690036, 'recall': 0.7429858991132432, 'f1': 0.688906860762906, 'number': 6879} | {'precision': 0.8048323036187114, 'recall': 0.8264415996374759, 'f1': 0.8154938237102454, 'number': 8827} | {'precision': 0.9815401570464252, 'recall': 0.9909596662030598, 'f1': 0.9862274205827393, 'number': 7190} | {'precision': 0.582089552238806, 'recall': 0.9512195121951219, 'f1': 0.7222222222222221, 'number': 82} | {'precision': 0.9313403416557161, 'recall': 0.9840333217632766, 'f1': 0.9569620253164556, 'number': 5762} | 0.8287 | 0.8796 | 0.8534 | 0.8332 |
| 0.4917 | 1.98 | 10000 | 0.4697 | {'precision': 0.6389365351629502, 'recall': 0.7581043756359936, 'f1': 0.6934379363074265, 'number': 6879} | {'precision': 0.8106822956983302, 'recall': 0.8305199954684491, 'f1': 0.8204812534974818, 'number': 8827} | {'precision': 0.9851553829078802, 'recall': 0.9876216968011127, 'f1': 0.9863869981941935, 'number': 7190} | {'precision': 0.6347826086956522, 'recall': 0.8902439024390244, 'f1': 0.7411167512690355, 'number': 82} | {'precision': 0.9327744904667982, 'recall': 0.9849010760152724, 'f1': 0.9581293263548878, 'number': 5762} | 0.8296 | 0.8836 | 0.8557 | 0.8341 |
| 0.4913 | 2.18 | 11000 | 0.4685 | {'precision': 0.6405220633934121, 'recall': 0.7490914377089694, 'f1': 0.6905655320289467, 'number': 6879} | {'precision': 0.8053573388955978, 'recall': 0.8310864393338621, 'f1': 0.8180196253345228, 'number': 8827} | {'precision': 0.9836745987825124, 'recall': 0.9888734353268428, 'f1': 0.9862671660424469, 'number': 7190} | {'precision': 0.6454545454545455, 'recall': 0.8658536585365854, 'f1': 0.7395833333333335, 'number': 82} | {'precision': 0.9313854235062377, 'recall': 0.9847275251648733, 'f1': 0.9573139868398851, 'number': 5762} | 0.8287 | 0.8818 | 0.8544 | 0.8355 |
| 0.4769 | 2.38 | 12000 | 0.4659 | {'precision': 0.6392910634048926, 'recall': 0.7445849687454572, 'f1': 0.6879323081055672, 'number': 6879} | {'precision': 0.8030103274005713, 'recall': 0.8280276424606321, 'f1': 0.8153271236544146, 'number': 8827} | {'precision': 0.9858431644691187, 'recall': 0.9878998609179416, 'f1': 0.9868704411253908, 'number': 7190} | {'precision': 0.6607142857142857, 'recall': 0.9024390243902439, 'f1': 0.7628865979381443, 'number': 82} | {'precision': 0.9313339888561127, 'recall': 0.9862894828184658, 'f1': 0.958024275118004, 'number': 5762} | 0.8283 | 0.8800 | 0.8534 | 0.8353 |
| 0.4752 | 2.57 | 13000 | 0.4651 | {'precision': 0.641625, 'recall': 0.7461840383776712, 'f1': 0.6899657235029236, 'number': 6879} | {'precision': 0.8089998899768952, 'recall': 0.833012348476266, 'f1': 0.8208305425318152, 'number': 8827} | {'precision': 0.9854389127721537, 'recall': 0.988317107093185, 'f1': 0.9868759113950422, 'number': 7190} | {'precision': 0.6634615384615384, 'recall': 0.8414634146341463, 'f1': 0.7419354838709676, 'number': 82} | {'precision': 0.932905772076961, 'recall': 0.9845539743144741, 'f1': 0.95803428185426, 'number': 5762} | 0.8310 | 0.8815 | 0.8555 | 0.8359 |
| 0.4834 | 2.77 | 14000 | 0.4628 | {'precision': 0.6457421533074903, 'recall': 0.7506905073411834, 'f1': 0.694272653939231, 'number': 6879} | {'precision': 0.8060932688077431, 'recall': 0.830293417922284, 'f1': 0.8180143981248955, 'number': 8827} | {'precision': 0.9835589941972921, 'recall': 0.990125173852573, 'f1': 0.9868311616301636, 'number': 7190} | {'precision': 0.6324786324786325, 'recall': 0.9024390243902439, 'f1': 0.7437185929648242, 'number': 82} | {'precision': 0.9318890530116527, 'recall': 0.9854217285664699, 'f1': 0.9579080556727119, 'number': 5762} | 0.8306 | 0.8825 | 0.8558 | 0.8365 |
| 0.4784 | 2.97 | 15000 | 0.4626 | {'precision': 0.6482109227871939, 'recall': 0.7505451373746184, 'f1': 0.6956345998383185, 'number': 6879} | {'precision': 0.8074424749532093, 'recall': 0.8308598617876969, 'f1': 0.8189838079285315, 'number': 8827} | {'precision': 0.9836881393419962, 'recall': 0.9897079276773296, 'f1': 0.9866888519134775, 'number': 7190} | {'precision': 0.6460176991150443, 'recall': 0.8902439024390244, 'f1': 0.7487179487179487, 'number': 82} | {'precision': 0.9323925172300623, 'recall': 0.9861159319680667, 'f1': 0.958502024291498, 'number': 5762} | 0.8320 | 0.8827 | 0.8566 | 0.8370 |
Framework versions
- Transformers 4.33.3
- Pytorch 2.0.1+cu118
- Datasets 2.14.5
- Tokenizers 0.13.3