bert-base-uncased-finetuned-math_punctuation
This model is a fine-tuned version of bert-base-uncased on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.1688
- Cls Report: {'EMPTY': {'precision': 0.9816733583732677, 'recall': 0.9870318722937851, 'f1-score': 0.9843453228066947, 'support': 92149}, '.': {'precision': 0.7987590486039297, 'recall': 0.7989242863053372, 'f1-score': 0.7988416589099182, 'support': 4834}, ',': {'precision': 0.6800739713361073, 'recall': 0.5517629407351838, 'f1-score': 0.6092358666390557, 'support': 2666}, '?': {'precision': 0.7534498620055198, 'recall': 0.7527573529411765, 'f1-score': 0.7531034482758621, 'support': 1088}, 'accuracy': 0.9639556468824761, 'macro avg': {'precision': 0.8034890600797061, 'recall': 0.7726191130688707, 'f1-score': 0.7863815741578827, 'support': 100737}, 'weighted avg': {'precision': 0.9624492510113833, 'recall': 0.9639556468824761, 'f1-score': 0.9630189215746797, 'support': 100737}}
- Precision: 0.8035
- Recall: 0.7726
- F Score: 0.7864
- Auc: 0.9378
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 12
Training results
| Training Loss | Epoch | Step | Validation Loss | Cls Report | Precision | Recall | F Score | Auc |
|---|---|---|---|---|---|---|---|---|
| 0.0872 | 0.61 | 500 | 0.0870 | {'EMPTY': {'precision': 0.9790177368052186, 'recall': 0.9853498138883764, 'f1-score': 0.9821735697210293, 'support': 92149}, '.': {'precision': 0.7199926021823562, 'recall': 0.8053371948696731, 'f1-score': 0.7602773166682941, 'support': 4834}, ',': {'precision': 0.647912885662432, 'recall': 0.40172543135783945, 'f1-score': 0.49594813614262556, 'support': 2666}, '?': {'precision': 0.7650214592274678, 'recall': 0.6553308823529411, 'f1-score': 0.7059405940594059, 'support': 1088}, 'accuracy': 0.9577017381895431, 'macro avg': {'precision': 0.7779861709693687, 'recall': 0.7119358306172076, 'f1-score': 0.7360849041478388, 'support': 100737}, 'weighted avg': {'precision': 0.9555141484124913, 'recall': 0.9577017381895431, 'f1-score': 0.9556742202198848, 'support': 100737}} | 0.7780 | 0.7119 | 0.7361 | 0.9357 |
| 0.0649 | 1.23 | 1000 | 0.0828 | {'EMPTY': {'precision': 0.9791280947255113, 'recall': 0.9871078362217712, 'f1-score': 0.9831017730438966, 'support': 92149}, '.': {'precision': 0.7804928131416837, 'recall': 0.7863053371948697, 'f1-score': 0.7833882934872218, 'support': 4834}, ',': {'precision': 0.6437346437346437, 'recall': 0.4913728432108027, 'f1-score': 0.557328228036588, 'support': 2666}, '?': {'precision': 0.7918454935622318, 'recall': 0.6783088235294118, 'f1-score': 0.7306930693069307, 'support': 1088}, 'accuracy': 0.9610173024807172, 'macro avg': {'precision': 0.7988002612910177, 'recall': 0.7357737100392139, 'f1-score': 0.7636278409686592, 'support': 100737}, 'weighted avg': {'precision': 0.9586974152176492, 'recall': 0.9610173024807172, 'f1-score': 0.9595240617676797, 'support': 100737}} | 0.7988 | 0.7358 | 0.7636 | 0.9422 |
| 0.0616 | 1.84 | 1500 | 0.0819 | {'EMPTY': {'precision': 0.9811004732986104, 'recall': 0.9852847019501025, 'f1-score': 0.9831881358593543, 'support': 92149}, '.': {'precision': 0.7373263236950808, 'recall': 0.8123707074886223, 'f1-score': 0.7730314960629922, 'support': 4834}, ',': {'precision': 0.661874334398296, 'recall': 0.4662415603900975, 'f1-score': 0.5470950704225351, 'support': 2666}, '?': {'precision': 0.7820383451059536, 'recall': 0.7123161764705882, 'f1-score': 0.7455507455507455, 'support': 1088}, 'accuracy': 0.9603025700586676, 'macro avg': {'precision': 0.7905848691244852, 'recall': 0.7440532865748527, 'f1-score': 0.7622163619739069, 'support': 100737}, 'weighted avg': {'precision': 0.9588043882358697, 'recall': 0.9603025700586676, 'f1-score': 0.9589957260210037, 'support': 100737}} | 0.7906 | 0.7441 | 0.7622 | 0.9428 |
| 0.0442 | 2.45 | 2000 | 0.0880 | {'EMPTY': {'precision': 0.9839689595583041, 'recall': 0.9824740366146133, 'f1-score': 0.9832209298537127, 'support': 92149}, '.': {'precision': 0.79424216765453, 'recall': 0.7761688043028547, 'f1-score': 0.7851014856664574, 'support': 4834}, ',': {'precision': 0.585063752276867, 'recall': 0.6024006001500375, 'f1-score': 0.5936056181851784, 'support': 2666}, '?': {'precision': 0.6926131850675139, 'recall': 0.8014705882352942, 'f1-score': 0.7430762675756285, 'support': 1088}, 'accuracy': 0.9605606678777411, 'macro avg': {'precision': 0.7639720161393038, 'recall': 0.7906285073257, 'f1-score': 0.7762510753202443, 'support': 100737}, 'weighted avg': {'precision': 0.9611608981973231, 'recall': 0.9605606678777411, 'f1-score': 0.9608090930244636, 'support': 100737}} | 0.7640 | 0.7906 | 0.7763 | 0.9480 |
| 0.0403 | 3.07 | 2500 | 0.1003 | {'EMPTY': {'precision': 0.9812742167010242, 'recall': 0.9866412006641417, 'f1-score': 0.9839503901472927, 'support': 92149}, '.': {'precision': 0.7884338341222473, 'recall': 0.7925113777410012, 'f1-score': 0.7904673475704116, 'support': 4834}, ',': {'precision': 0.6488584474885845, 'recall': 0.5330082520630157, 'f1-score': 0.5852553542009885, 'support': 2666}, '?': {'precision': 0.7584541062801933, 'recall': 0.7215073529411765, 'f1-score': 0.7395195478097033, 'support': 1088}, 'accuracy': 0.9624566941640113, 'macro avg': {'precision': 0.7942551511480123, 'recall': 0.7584170458523338, 'f1-score': 0.7747981599320991, 'support': 100737}, 'weighted avg': {'precision': 0.9608165980480563, 'recall': 0.9624566941640113, 'f1-score': 0.9614744503226722, 'support': 100737}} | 0.7943 | 0.7584 | 0.7748 | 0.9393 |
| 0.0309 | 3.68 | 3000 | 0.0944 | {'EMPTY': {'precision': 0.981867888441049, 'recall': 0.9860660452093891, 'f1-score': 0.9839624889004396, 'support': 92149}, '.': {'precision': 0.7817371937639198, 'recall': 0.7987174182871328, 'f1-score': 0.790136089225417, 'support': 4834}, ',': {'precision': 0.659784138901924, 'recall': 0.5273818454613654, 'f1-score': 0.5861997081509277, 'support': 2666}, '?': {'precision': 0.7339857651245552, 'recall': 0.7582720588235294, 'f1-score': 0.7459312839059674, 'support': 1088}, 'accuracy': 0.9624765478424016, 'macro avg': {'precision': 0.7893437465578619, 'recall': 0.7676093419453542, 'f1-score': 0.7765573925456879, 'support': 100737}, 'weighted avg': {'precision': 0.9610631910159828, 'recall': 0.9624765478424016, 'f1-score': 0.9615638633652216, 'support': 100737}} | 0.7893 | 0.7676 | 0.7766 | 0.9450 |
| 0.0188 | 4.29 | 3500 | 0.1144 | {'EMPTY': {'precision': 0.9802141226142778, 'recall': 0.9876070277485377, 'f1-score': 0.9838966879827886, 'support': 92149}, '.': {'precision': 0.7962216090002122, 'recall': 0.7759619362846504, 'f1-score': 0.7859612362493452, 'support': 4834}, ',': {'precision': 0.6711409395973155, 'recall': 0.5251312828207052, 'f1-score': 0.5892255892255893, 'support': 2666}, '?': {'precision': 0.7253649635036497, 'recall': 0.7306985294117647, 'f1-score': 0.728021978021978, 'support': 1088}, 'accuracy': 0.962436840485621, 'macro avg': {'precision': 0.7932354086788638, 'recall': 0.7548496940664144, 'f1-score': 0.7717763728699254, 'support': 100737}, 'weighted avg': {'precision': 0.9604529146981596, 'recall': 0.962436840485621, 'f1-score': 0.9611899882855223, 'support': 100737}} | 0.7932 | 0.7548 | 0.7718 | 0.9384 |
| 0.0201 | 4.91 | 4000 | 0.1079 | {'EMPTY': {'precision': 0.9820999805317009, 'recall': 0.9853932218472257, 'f1-score': 0.9837438450329619, 'support': 92149}, '.': {'precision': 0.7719614921780987, 'recall': 0.7962350020686801, 'f1-score': 0.7839103869653768, 'support': 4834}, ',': {'precision': 0.6680672268907563, 'recall': 0.5367591897974494, 'f1-score': 0.5952579034941764, 'support': 2666}, '?': {'precision': 0.7176368375325803, 'recall': 0.7591911764705882, 'f1-score': 0.7378293881196962, 'support': 1088}, 'accuracy': 0.9620000595610352, 'macro avg': {'precision': 0.7849413842832841, 'recall': 0.7693946475459859, 'f1-score': 0.7751853809030529, 'support': 100737}, 'weighted avg': {'precision': 0.9608490332780492, 'recall': 0.9620000595610352, 'f1-score': 0.961217331581472, 'support': 100737}} | 0.7849 | 0.7694 | 0.7752 | 0.9455 |
| 0.0125 | 5.52 | 4500 | 0.1295 | {'EMPTY': {'precision': 0.9802362170783223, 'recall': 0.9871186882114836, 'f1-score': 0.9836654140420125, 'support': 92149}, '.': {'precision': 0.7850408548082967, 'recall': 0.7751344642118329, 'f1-score': 0.7800562090142604, 'support': 4834}, ',': {'precision': 0.6598960793575814, 'recall': 0.5240060015003751, 'f1-score': 0.5841522057286221, 'support': 2666}, '?': {'precision': 0.7573739295908658, 'recall': 0.7316176470588235, 'f1-score': 0.7442730247779334, 'support': 1088}, 'accuracy': 0.9619305716866693, 'macro avg': {'precision': 0.7956367702087666, 'recall': 0.7544692002456288, 'f1-score': 0.773036713390707, 'support': 100737}, 'weighted avg': {'precision': 0.9599847170618124, 'recall': 0.9619305716866693, 'f1-score': 0.9607363211567075, 'support': 100737}} | 0.7956 | 0.7545 | 0.7730 | 0.9359 |
| 0.011 | 6.13 | 5000 | 0.1329 | {'EMPTY': {'precision': 0.9830135739743687, 'recall': 0.9847203984850622, 'f1-score': 0.9838662459746934, 'support': 92149}, '.': {'precision': 0.7825380710659898, 'recall': 0.7972693421597021, 'f1-score': 0.789835024080336, 'support': 4834}, ',': {'precision': 0.629092416079569, 'recall': 0.5693923480870218, 'f1-score': 0.5977554636739515, 'support': 2666}, '?': {'precision': 0.7495412844036697, 'recall': 0.7509191176470589, 'f1-score': 0.7502295684113865, 'support': 1088}, 'accuracy': 0.962208523184133, 'macro avg': {'precision': 0.7860463363808993, 'recall': 0.7755753015947113, 'f1-score': 0.7804215755350918, 'support': 100737}, 'weighted avg': {'precision': 0.9615053869223467, 'recall': 0.962208523184133, 'f1-score': 0.9618136240240696, 'support': 100737}} | 0.7860 | 0.7756 | 0.7804 | 0.9398 |
| 0.0089 | 6.75 | 5500 | 0.1353 | {'EMPTY': {'precision': 0.9801606962066217, 'recall': 0.9875744717794007, 'f1-score': 0.9838536176653423, 'support': 92149}, '.': {'precision': 0.7881303174932559, 'recall': 0.7856847331402566, 'f1-score': 0.7869056251942402, 'support': 4834}, ',': {'precision': 0.6794234592445328, 'recall': 0.5127531882970743, 'f1-score': 0.5844377939290295, 'support': 2666}, '?': {'precision': 0.7528301886792453, 'recall': 0.7334558823529411, 'f1-score': 0.7430167597765363, 'support': 1088}, 'accuracy': 0.9625758162343528, 'macro avg': {'precision': 0.800136165405914, 'recall': 0.7548670688924182, 'f1-score': 0.7745534491412871, 'support': 100737}, 'weighted avg': {'precision': 0.9605316034538982, 'recall': 0.9625758162343528, 'f1-score': 0.961231148432892, 'support': 100737}} | 0.8001 | 0.7549 | 0.7746 | 0.9347 |
| 0.0058 | 7.36 | 6000 | 0.1412 | {'EMPTY': {'precision': 0.9816272115893218, 'recall': 0.9868256844892511, 'f1-score': 0.9842195837346985, 'support': 92149}, '.': {'precision': 0.8054963783553473, 'recall': 0.782167976830782, 'f1-score': 0.7936607892527289, 'support': 4834}, ',': {'precision': 0.6578260869565218, 'recall': 0.567516879219805, 'f1-score': 0.6093435360451067, 'support': 2666}, '?': {'precision': 0.7368896925858951, 'recall': 0.7490808823529411, 'f1-score': 0.7429352780309937, 'support': 1088}, 'accuracy': 0.963340182852378, 'macro avg': {'precision': 0.7954598423717714, 'recall': 0.7713978557231947, 'f1-score': 0.782539796765882, 'support': 100737}, 'weighted avg': {'precision': 0.9619626924275458, 'recall': 0.963340182852378, 'f1-score': 0.9625483201446381, 'support': 100737}} | 0.7955 | 0.7714 | 0.7825 | 0.9414 |
| 0.0062 | 7.98 | 6500 | 0.1473 | {'EMPTY': {'precision': 0.981030745505721, 'recall': 0.9872055041291821, 'f1-score': 0.9841084390787439, 'support': 92149}, '.': {'precision': 0.7817779565567177, 'recall': 0.8040959867604468, 'f1-score': 0.7927799306547011, 'support': 4834}, ',': {'precision': 0.6930792377131394, 'recall': 0.5183795948987246, 'f1-score': 0.5931330472103004, 'support': 2666}, '?': {'precision': 0.761996161228407, 'recall': 0.7297794117647058, 'f1-score': 0.7455399061032865, 'support': 1088}, 'accuracy': 0.9632309876212315, 'macro avg': {'precision': 0.8044710252509962, 'recall': 0.7598651243882648, 'f1-score': 0.778890330761758, 'support': 100737}, 'weighted avg': {'precision': 0.9614830487384137, 'recall': 0.9632309876212315, 'f1-score': 0.9620035027760906, 'support': 100737}} | 0.8045 | 0.7599 | 0.7789 | 0.9366 |
| 0.0042 | 8.59 | 7000 | 0.1567 | {'EMPTY': {'precision': 0.9827906976744186, 'recall': 0.9860009332711153, 'f1-score': 0.9843931982296761, 'support': 92149}, '.': {'precision': 0.8073005093378608, 'recall': 0.7869259412494828, 'f1-score': 0.7969830295411691, 'support': 4834}, ',': {'precision': 0.6395010395010395, 'recall': 0.5768942235558889, 'f1-score': 0.6065864720962334, 'support': 2666}, '?': {'precision': 0.7170940170940171, 'recall': 0.7711397058823529, 'f1-score': 0.7431355181576615, 'support': 1088}, 'accuracy': 0.9633004754955975, 'macro avg': {'precision': 0.7866715659018341, 'recall': 0.7802402009897099, 'f1-score': 0.782774554506185, 'support': 100737}, 'weighted avg': {'precision': 0.9624147902364303, 'recall': 0.9633004754955975, 'f1-score': 0.9627957529689442, 'support': 100737}} | 0.7867 | 0.7802 | 0.7828 | 0.9384 |
| 0.0036 | 9.2 | 7500 | 0.1570 | {'EMPTY': {'precision': 0.9826980264936469, 'recall': 0.9861745651065122, 'f1-score': 0.9844332264494322, 'support': 92149}, '.': {'precision': 0.7967748604506926, 'recall': 0.7972693421597021, 'f1-score': 0.7970220246096578, 'support': 4834}, ',': {'precision': 0.6540284360189573, 'recall': 0.5693923480870218, 'f1-score': 0.6087828353719671, 'support': 2666}, '?': {'precision': 0.7427536231884058, 'recall': 0.7536764705882353, 'f1-score': 0.7481751824817517, 'support': 1088}, 'accuracy': 0.963568500153866, 'macro avg': {'precision': 0.7940637365379256, 'recall': 0.7766281814853679, 'f1-score': 0.7846033172282022, 'support': 100737}, 'weighted avg': {'precision': 0.9624865329644247, 'recall': 0.963568500153866, 'f1-score': 0.9629467969930973, 'support': 100737}} | 0.7941 | 0.7766 | 0.7846 | 0.9388 |
| 0.0026 | 9.82 | 8000 | 0.1661 | {'EMPTY': {'precision': 0.9815270404419699, 'recall': 0.9871403921909082, 'f1-score': 0.9843257135127824, 'support': 92149}, '.': {'precision': 0.8012995179207714, 'recall': 0.7908564335953662, 'f1-score': 0.7960437272254035, 'support': 4834}, ',': {'precision': 0.6715935334872979, 'recall': 0.5453863465866466, 'f1-score': 0.6019457669219622, 'support': 2666}, '?': {'precision': 0.7368888888888889, 'recall': 0.7619485294117647, 'f1-score': 0.7492092182557614, 'support': 1088}, 'accuracy': 0.9635982806714514, 'macro avg': {'precision': 0.7978272451847321, 'recall': 0.7713329254461714, 'f1-score': 0.7828811064789774, 'support': 100737}, 'weighted avg': {'precision': 0.9620340152149095, 'recall': 0.9635982806714514, 'f1-score': 0.96263173010883, 'support': 100737}} | 0.7978 | 0.7713 | 0.7829 | 0.9351 |
| 0.0023 | 10.43 | 8500 | 0.1675 | {'EMPTY': {'precision': 0.9818959095767076, 'recall': 0.9870318722937851, 'f1-score': 0.984457192336833, 'support': 92149}, '.': {'precision': 0.7958974358974359, 'recall': 0.8026479106330161, 'f1-score': 0.7992584200226595, 'support': 4834}, ',': {'precision': 0.6728025770823746, 'recall': 0.5483870967741935, 'f1-score': 0.6042570779086587, 'support': 2666}, '?': {'precision': 0.7608695652173914, 'recall': 0.7398897058823529, 'f1-score': 0.750232991612302, 'support': 1088}, 'accuracy': 0.9639060126865004, 'macro avg': {'precision': 0.8028663719434774, 'recall': 0.7694891463958369, 'f1-score': 0.7845514204701133, 'support': 100737}, 'weighted avg': {'precision': 0.9624032096863155, 'recall': 0.9639060126865004, 'f1-score': 0.9629784873841293, 'support': 100737}} | 0.8029 | 0.7695 | 0.7846 | 0.9363 |
| 0.0023 | 11.04 | 9000 | 0.1693 | {'EMPTY': {'precision': 0.9820605039475532, 'recall': 0.9867497205612649, 'f1-score': 0.9843995279801664, 'support': 92149}, '.': {'precision': 0.7992526468756488, 'recall': 0.7964418700868846, 'f1-score': 0.7978447829240494, 'support': 4834}, ',': {'precision': 0.6733121884911645, 'recall': 0.5573893473368342, 'f1-score': 0.6098912374307408, 'support': 2666}, '?': {'precision': 0.7384341637010676, 'recall': 0.7628676470588235, 'f1-score': 0.7504520795660037, 'support': 1088}, 'accuracy': 0.9638365248121346, 'macro avg': {'precision': 0.7982648757538585, 'recall': 0.7758621462609517, 'f1-score': 0.7856469069752401, 'support': 100737}, 'weighted avg': {'precision': 0.962485951913241, 'recall': 0.9638365248121346, 'f1-score': 0.9630093777465815, 'support': 100737}} | 0.7983 | 0.7759 | 0.7856 | 0.9380 |
| 0.0019 | 11.66 | 9500 | 0.1688 | {'EMPTY': {'precision': 0.9816733583732677, 'recall': 0.9870318722937851, 'f1-score': 0.9843453228066947, 'support': 92149}, '.': {'precision': 0.7987590486039297, 'recall': 0.7989242863053372, 'f1-score': 0.7988416589099182, 'support': 4834}, ',': {'precision': 0.6800739713361073, 'recall': 0.5517629407351838, 'f1-score': 0.6092358666390557, 'support': 2666}, '?': {'precision': 0.7534498620055198, 'recall': 0.7527573529411765, 'f1-score': 0.7531034482758621, 'support': 1088}, 'accuracy': 0.9639556468824761, 'macro avg': {'precision': 0.8034890600797061, 'recall': 0.7726191130688707, 'f1-score': 0.7863815741578827, 'support': 100737}, 'weighted avg': {'precision': 0.9624492510113833, 'recall': 0.9639556468824761, 'f1-score': 0.9630189215746797, 'support': 100737}} | 0.8035 | 0.7726 | 0.7864 | 0.9378 |
Framework versions
- Transformers 4.25.1
- Pytorch 2.0.0.dev20230111
- Datasets 2.8.0
- Tokenizers 0.13.2