nyt-ingredient-tagger-gte-small
This model is a fine-tuned version of thenlper/gte-small on the nyt_ingredients dataset. It achieves the following results on the evaluation set:
- Loss: 0.8647
- Comment: {'precision': 0.700344295381693, 'recall': 0.8211302895322939, 'f1': 0.7559428461587748, 'number': 7184}
- Name: {'precision': 0.8125, 'recall': 0.8314522197140707, 'f1': 0.8218668650055783, 'number': 9303}
- Qty: {'precision': 0.987037037037037, 'recall': 0.9920233980324382, 'f1': 0.9895239358175308, 'number': 7522}
- Range End: {'precision': 0.7394366197183099, 'recall': 0.9375, 'f1': 0.8267716535433071, 'number': 112}
- Unit: {'precision': 0.9283142901862577, 'recall': 0.9870194707938093, 'f1': 0.9567672205194386, 'number': 6009}
- Overall Precision: 0.8470
- Overall Recall: 0.9005
- Overall F1: 0.8729
- Overall Accuracy: 0.8468
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5
- label_smoothing_factor: 0.1
Training results
| Training Loss | Epoch | Step | Validation Loss | Comment | Name | Qty | Range End | Unit | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.9427 | 0.2 | 1000 | 0.9321 | {'precision': 0.6076682497393722, 'recall': 0.7789161098737936, 'f1': 0.6827173347214992, 'number': 6735} | {'precision': 0.7829228243021347, 'recall': 0.8126349278491081, 'f1': 0.7975022301516503, 'number': 8801} | {'precision': 0.97529493407356, 'recall': 0.9913939051918735, 'f1': 0.9832785279507451, 'number': 7088} | {'precision': 0.52046783625731, 'recall': 0.978021978021978, 'f1': 0.6793893129770994, 'number': 91} | {'precision': 0.914435009797518, 'recall': 0.982973494821836, 'f1': 0.9474663734032653, 'number': 5697} | 0.8032 | 0.8839 | 0.8416 | 0.8183 |
| 0.9169 | 0.4 | 2000 | 0.9112 | {'precision': 0.6358395387941388, 'recall': 0.7860430586488493, 'f1': 0.7030077684084721, 'number': 6735} | {'precision': 0.781960613643782, 'recall': 0.8166117486649245, 'f1': 0.798910626945309, 'number': 8801} | {'precision': 0.9746502285635129, 'recall': 0.9926636568848759, 'f1': 0.9835744740336898, 'number': 7088} | {'precision': 0.6277372262773723, 'recall': 0.945054945054945, 'f1': 0.7543859649122806, 'number': 91} | {'precision': 0.9176201372997712, 'recall': 0.9854309285588907, 'f1': 0.9503173931443081, 'number': 5697} | 0.8137 | 0.8875 | 0.8490 | 0.8256 |
| 0.9 | 0.59 | 3000 | 0.9021 | {'precision': 0.6538183570224042, 'recall': 0.8059391239792131, 'f1': 0.7219525171244263, 'number': 6735} | {'precision': 0.7920824651825858, 'recall': 0.8207021929326213, 'f1': 0.8061383928571427, 'number': 8801} | {'precision': 0.9837443946188341, 'recall': 0.9904063205417607, 'f1': 0.9870641169853768, 'number': 7088} | {'precision': 0.6792452830188679, 'recall': 0.7912087912087912, 'f1': 0.7309644670050762, 'number': 91} | {'precision': 0.9138857235878235, 'recall': 0.9854309285588907, 'f1': 0.9483108108108108, 'number': 5697} | 0.8231 | 0.8925 | 0.8564 | 0.8302 |
| 0.9061 | 0.79 | 4000 | 0.8912 | {'precision': 0.6613263785394933, 'recall': 0.7906458797327395, 'f1': 0.7202272266179753, 'number': 6735} | {'precision': 0.7952695269526953, 'recall': 0.821383933643904, 'f1': 0.8081158124196521, 'number': 8801} | {'precision': 0.985236220472441, 'recall': 0.9885722347629797, 'f1': 0.9869014084507043, 'number': 7088} | {'precision': 0.656, 'recall': 0.9010989010989011, 'f1': 0.7592592592592593, 'number': 91} | {'precision': 0.9215106732348112, 'recall': 0.9850798665964543, 'f1': 0.9522355137015356, 'number': 5697} | 0.8289 | 0.8889 | 0.8578 | 0.8326 |
| 0.8889 | 0.99 | 5000 | 0.8908 | {'precision': 0.6653019447287615, 'recall': 0.7720861172976986, 'f1': 0.7147275101367603, 'number': 6735} | {'precision': 0.7959925134867335, 'recall': 0.8214975570957845, 'f1': 0.8085439498993513, 'number': 8801} | {'precision': 0.9857926571951048, 'recall': 0.9887133182844243, 'f1': 0.9872508276396421, 'number': 7088} | {'precision': 0.6439393939393939, 'recall': 0.9340659340659341, 'f1': 0.7623318385650225, 'number': 91} | {'precision': 0.9227729293594599, 'recall': 0.9836756187467088, 'f1': 0.9522514868309261, 'number': 5697} | 0.8317 | 0.8844 | 0.8572 | 0.8311 |
| 0.88 | 1.19 | 6000 | 0.8873 | {'precision': 0.660211910851297, 'recall': 0.8048997772828508, 'f1': 0.7254114813327982, 'number': 6735} | {'precision': 0.7929331723910896, 'recall': 0.8210430632882627, 'f1': 0.8067433292397008, 'number': 8801} | {'precision': 0.9827899818105499, 'recall': 0.9909706546275395, 'f1': 0.9868633649455567, 'number': 7088} | {'precision': 0.635036496350365, 'recall': 0.9560439560439561, 'f1': 0.7631578947368421, 'number': 91} | {'precision': 0.9224067072168338, 'recall': 0.984904335615236, 'f1': 0.9526315789473684, 'number': 5697} | 0.8266 | 0.8929 | 0.8585 | 0.8338 |
| 0.8751 | 1.39 | 7000 | 0.8866 | {'precision': 0.6715582638975252, 'recall': 0.8017817371937639, 'f1': 0.7309149972929073, 'number': 6735} | {'precision': 0.8020833333333334, 'recall': 0.8224065447108283, 'f1': 0.8121178120617111, 'number': 8801} | {'precision': 0.9849719101123595, 'recall': 0.9894187358916479, 'f1': 0.9871903153153153, 'number': 7088} | {'precision': 0.6258992805755396, 'recall': 0.9560439560439561, 'f1': 0.7565217391304349, 'number': 91} | {'precision': 0.9254720105995363, 'recall': 0.9808671230472178, 'f1': 0.9523647209203238, 'number': 5697} | 0.8341 | 0.8914 | 0.8618 | 0.8348 |
| 0.8816 | 1.58 | 8000 | 0.8823 | {'precision': 0.6672340425531915, 'recall': 0.8148478099480326, 'f1': 0.7336898395721925, 'number': 6735} | {'precision': 0.797549398388343, 'recall': 0.8209294398363822, 'f1': 0.8090705487122061, 'number': 8801} | {'precision': 0.9851018973998594, 'recall': 0.988854401805869, 'f1': 0.9869745828346124, 'number': 7088} | {'precision': 0.6439393939393939, 'recall': 0.9340659340659341, 'f1': 0.7623318385650225, 'number': 91} | {'precision': 0.9253386190948133, 'recall': 0.9833245567842724, 'f1': 0.9534507701472215, 'number': 5697} | 0.8308 | 0.8943 | 0.8614 | 0.8360 |
| 0.8756 | 1.78 | 9000 | 0.8817 | {'precision': 0.6767485822306238, 'recall': 0.7973273942093542, 'f1': 0.7321063394683027, 'number': 6735} | {'precision': 0.8019714254070218, 'recall': 0.8227474150664698, 'f1': 0.8122265844083006, 'number': 8801} | {'precision': 0.9834641255605381, 'recall': 0.9901241534988713, 'f1': 0.9867829021372329, 'number': 7088} | {'precision': 0.7043478260869566, 'recall': 0.8901098901098901, 'f1': 0.7864077669902914, 'number': 91} | {'precision': 0.9209706509263814, 'recall': 0.9859575215025452, 'f1': 0.9523567310952866, 'number': 5697} | 0.8355 | 0.8914 | 0.8625 | 0.8375 |
| 0.8695 | 1.98 | 10000 | 0.8788 | {'precision': 0.6812743986903412, 'recall': 0.8032665181885672, 'f1': 0.7372581084764241, 'number': 6735} | {'precision': 0.7975521005623553, 'recall': 0.821838427451426, 'f1': 0.8095131505316173, 'number': 8801} | {'precision': 0.9779074614422676, 'recall': 0.9929458239277652, 'f1': 0.9853692684634231, 'number': 7088} | {'precision': 0.6267605633802817, 'recall': 0.978021978021978, 'f1': 0.7639484978540774, 'number': 91} | {'precision': 0.9206036745406824, 'recall': 0.9850798665964543, 'f1': 0.9517510387518019, 'number': 5697} | 0.8337 | 0.8934 | 0.8625 | 0.8376 |
| 0.8537 | 2.18 | 11000 | 0.8804 | {'precision': 0.6863314805457301, 'recall': 0.8066815144766147, 'f1': 0.741655859668282, 'number': 6735} | {'precision': 0.8018565587357719, 'recall': 0.8244517668446767, 'f1': 0.8129971988795517, 'number': 8801} | {'precision': 0.9830627099664053, 'recall': 0.9908295711060948, 'f1': 0.9869308600337269, 'number': 7088} | {'precision': 0.6825396825396826, 'recall': 0.945054945054945, 'f1': 0.792626728110599, 'number': 91} | {'precision': 0.9248306624814142, 'recall': 0.9826224328593997, 'f1': 0.9528510638297872, 'number': 5697} | 0.8385 | 0.8938 | 0.8653 | 0.8384 |
| 0.854 | 2.38 | 12000 | 0.8817 | {'precision': 0.6863779033270558, 'recall': 0.8117297698589458, 'f1': 0.7438095238095238, 'number': 6735} | {'precision': 0.8055925432756325, 'recall': 0.8249062606521986, 'f1': 0.8151350137539999, 'number': 8801} | {'precision': 0.9833426651735723, 'recall': 0.9911117381489842, 'f1': 0.9872119168071951, 'number': 7088} | {'precision': 0.6742424242424242, 'recall': 0.978021978021978, 'f1': 0.7982062780269058, 'number': 91} | {'precision': 0.9206687428290444, 'recall': 0.9859575215025452, 'f1': 0.9521952873368368, 'number': 5697} | 0.8387 | 0.8960 | 0.8664 | 0.8390 |
| 0.8582 | 2.57 | 13000 | 0.8746 | {'precision': 0.6879990019960079, 'recall': 0.8188567186340014, 'f1': 0.7477459155311504, 'number': 6735} | {'precision': 0.8027563395810364, 'recall': 0.8272923531416885, 'f1': 0.8148396844049018, 'number': 8801} | {'precision': 0.9822476935979871, 'recall': 0.9913939051918735, 'f1': 0.9867996067967982, 'number': 7088} | {'precision': 0.6693548387096774, 'recall': 0.9120879120879121, 'f1': 0.772093023255814, 'number': 91} | {'precision': 0.9260487481346377, 'recall': 0.9803405301035633, 'f1': 0.9524215552523875, 'number': 5697} | 0.8387 | 0.8972 | 0.8669 | 0.8402 |
| 0.8554 | 2.77 | 14000 | 0.8743 | {'precision': 0.6870807453416149, 'recall': 0.8212323682256867, 'f1': 0.7481907338518768, 'number': 6735} | {'precision': 0.8064945140197274, 'recall': 0.8268378593341665, 'f1': 0.8165394973070018, 'number': 8801} | {'precision': 0.9841692350798543, 'recall': 0.9911117381489842, 'f1': 0.9876282862364684, 'number': 7088} | {'precision': 0.672, 'recall': 0.9230769230769231, 'f1': 0.7777777777777778, 'number': 91} | {'precision': 0.923507155782201, 'recall': 0.9854309285588907, 'f1': 0.9534646739130435, 'number': 5697} | 0.8394 | 0.8986 | 0.8680 | 0.8416 |
| 0.86 | 2.97 | 15000 | 0.8735 | {'precision': 0.6898646083765658, 'recall': 0.8095025983667409, 'f1': 0.7449105068998496, 'number': 6735} | {'precision': 0.8031156778256546, 'recall': 0.8259288717191229, 'f1': 0.8143625364104862, 'number': 8801} | {'precision': 0.9833473271760426, 'recall': 0.9913939051918735, 'f1': 0.9873542222846705, 'number': 7088} | {'precision': 0.6611570247933884, 'recall': 0.8791208791208791, 'f1': 0.7547169811320755, 'number': 91} | {'precision': 0.9261034881798644, 'recall': 0.9833245567842724, 'f1': 0.9538566320449514, 'number': 5697} | 0.8401 | 0.8950 | 0.8667 | 0.8423 |
| 0.845 | 3.17 | 16000 | 0.8782 | {'precision': 0.7035061991734436, 'recall': 0.7835189309576838, 'f1': 0.7413599325653274, 'number': 6735} | {'precision': 0.7948126167710737, 'recall': 0.8217248039995455, 'f1': 0.8080446927374302, 'number': 8801} | {'precision': 0.982394858180802, 'recall': 0.9919582392776524, 'f1': 0.9871533871533871, 'number': 7088} | {'precision': 0.719626168224299, 'recall': 0.8461538461538461, 'f1': 0.7777777777777778, 'number': 91} | {'precision': 0.9267119880616813, 'recall': 0.981042654028436, 'f1': 0.9531036834924966, 'number': 5697} | 0.8432 | 0.8872 | 0.8646 | 0.8397 |
| 0.846 | 3.37 | 17000 | 0.8759 | {'precision': 0.6963411491883535, 'recall': 0.8025241276911655, 'f1': 0.7456715182451542, 'number': 6735} | {'precision': 0.7992290748898678, 'recall': 0.8245653902965572, 'f1': 0.8116995693753145, 'number': 8801} | {'precision': 0.984164798206278, 'recall': 0.9908295711060948, 'f1': 0.9874859392575928, 'number': 7088} | {'precision': 0.6694214876033058, 'recall': 0.8901098901098901, 'f1': 0.7641509433962264, 'number': 91} | {'precision': 0.9245874587458746, 'recall': 0.9835000877654906, 'f1': 0.9531343029684443, 'number': 5697} | 0.8412 | 0.8929 | 0.8663 | 0.8403 |
| 0.8392 | 3.56 | 18000 | 0.8759 | {'precision': 0.7022442588726514, 'recall': 0.799109131403118, 'f1': 0.7475519133273144, 'number': 6735} | {'precision': 0.8000660211267606, 'recall': 0.8261561186228837, 'f1': 0.8129017832187377, 'number': 8801} | {'precision': 0.9823997765050985, 'recall': 0.9922404063205418, 'f1': 0.987295570997403, 'number': 7088} | {'precision': 0.6890756302521008, 'recall': 0.9010989010989011, 'f1': 0.7809523809523808, 'number': 91} | {'precision': 0.9239380968060587, 'recall': 0.9850798665964543, 'f1': 0.9535298615240847, 'number': 5697} | 0.8431 | 0.8933 | 0.8675 | 0.8409 |
| 0.8375 | 3.76 | 19000 | 0.8780 | {'precision': 0.6947714900620332, 'recall': 0.8148478099480326, 'f1': 0.7500341670083366, 'number': 6735} | {'precision': 0.8054323725055432, 'recall': 0.8254743779116009, 'f1': 0.8153302283822457, 'number': 8801} | {'precision': 0.9837648705388383, 'recall': 0.991676072234763, 'f1': 0.9877046300850137, 'number': 7088} | {'precision': 0.6864406779661016, 'recall': 0.8901098901098901, 'f1': 0.7751196172248803, 'number': 91} | {'precision': 0.9236477572559367, 'recall': 0.9831490258030542, 'f1': 0.9524700280588385, 'number': 5697} | 0.8419 | 0.8962 | 0.8682 | 0.8413 |
| 0.8366 | 3.96 | 20000 | 0.8742 | {'precision': 0.7003211303789338, 'recall': 0.8095025983667409, 'f1': 0.7509641873278236, 'number': 6735} | {'precision': 0.804251550044287, 'recall': 0.8253607544597205, 'f1': 0.8146694330735154, 'number': 8801} | {'precision': 0.9832073887489504, 'recall': 0.9912528216704289, 'f1': 0.9872137136433891, 'number': 7088} | {'precision': 0.7053571428571429, 'recall': 0.8681318681318682, 'f1': 0.7783251231527095, 'number': 91} | {'precision': 0.9234189723320159, 'recall': 0.9842022116903634, 'f1': 0.9528422125924038, 'number': 5697} | 0.8435 | 0.8950 | 0.8685 | 0.8417 |
| 0.8189 | 4.16 | 21000 | 0.8799 | {'precision': 0.7027131284557027, 'recall': 0.8114328136599851, 'f1': 0.7531697905181918, 'number': 6735} | {'precision': 0.8056294326241135, 'recall': 0.8260424951710033, 'f1': 0.8157082748948107, 'number': 8801} | {'precision': 0.9825345815285734, 'recall': 0.9920993227990971, 'f1': 0.9872937872937874, 'number': 7088} | {'precision': 0.6864406779661016, 'recall': 0.8901098901098901, 'f1': 0.7751196172248803, 'number': 91} | {'precision': 0.9251195777667821, 'recall': 0.9845532736527998, 'f1': 0.9539115646258504, 'number': 5697} | 0.8447 | 0.8960 | 0.8696 | 0.8425 |
| 0.8269 | 4.36 | 22000 | 0.8781 | {'precision': 0.6995086630462891, 'recall': 0.8032665181885672, 'f1': 0.7478056534660309, 'number': 6735} | {'precision': 0.8004192409532216, 'recall': 0.8243381433927963, 'f1': 0.8122026308424294, 'number': 8801} | {'precision': 0.9833403331933361, 'recall': 0.9909706546275395, 'f1': 0.987140749068934, 'number': 7088} | {'precision': 0.7079646017699115, 'recall': 0.8791208791208791, 'f1': 0.7843137254901961, 'number': 91} | {'precision': 0.9244473771032663, 'recall': 0.9836756187467088, 'f1': 0.9531422740028915, 'number': 5697} | 0.8425 | 0.8930 | 0.8670 | 0.8418 |
| 0.829 | 4.55 | 23000 | 0.8794 | {'precision': 0.70192058136517, 'recall': 0.8031180400890868, 'f1': 0.7491170971539367, 'number': 6735} | {'precision': 0.7987892129884425, 'recall': 0.8245653902965572, 'f1': 0.8114726601811473, 'number': 8801} | {'precision': 0.9836226203807391, 'recall': 0.9913939051918735, 'f1': 0.9874929735806632, 'number': 7088} | {'precision': 0.6991150442477876, 'recall': 0.8681318681318682, 'f1': 0.7745098039215685, 'number': 91} | {'precision': 0.9249050998514606, 'recall': 0.9836756187467088, 'f1': 0.9533855052739026, 'number': 5697} | 0.8429 | 0.8931 | 0.8673 | 0.8422 |
| 0.8183 | 4.75 | 24000 | 0.8815 | {'precision': 0.6988397296952696, 'recall': 0.8138084632516703, 'f1': 0.7519550006859651, 'number': 6735} | {'precision': 0.806115665854199, 'recall': 0.826724235882286, 'f1': 0.8162898973467212, 'number': 8801} | {'precision': 0.9834757036829576, 'recall': 0.9908295711060948, 'f1': 0.9871389415981446, 'number': 7088} | {'precision': 0.6837606837606838, 'recall': 0.8791208791208791, 'f1': 0.7692307692307692, 'number': 91} | {'precision': 0.9244473771032663, 'recall': 0.9836756187467088, 'f1': 0.9531422740028915, 'number': 5697} | 0.8435 | 0.8962 | 0.8691 | 0.8428 |
| 0.8201 | 4.95 | 25000 | 0.8808 | {'precision': 0.7003986112897004, 'recall': 0.8087602078693392, 'f1': 0.7506890848952591, 'number': 6735} | {'precision': 0.8026315789473685, 'recall': 0.8247926372003181, 'f1': 0.8135612216307089, 'number': 8801} | {'precision': 0.9834849545136459, 'recall': 0.9913939051918735, 'f1': 0.9874235930583853, 'number': 7088} | {'precision': 0.6923076923076923, 'recall': 0.8901098901098901, 'f1': 0.7788461538461537, 'number': 91} | {'precision': 0.9247400561148704, 'recall': 0.9835000877654906, 'f1': 0.9532153793807416, 'number': 5697} | 0.8432 | 0.8946 | 0.8682 | 0.8429 |
Framework versions
- Transformers 4.34.0
- Pytorch 2.0.1+cu118
- Datasets 2.14.5
- Tokenizers 0.14.1