This model is a fine-tuned version of microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext on the PubMed200kRCT dataset. It achieves the following results on the evaluation set:
More information needed
The model can be used for text classification tasks of Randomized Controlled Trials that does not have any structure. The text can be classified as one of the following:
The model can be directly used like this:
from transformers import TextClassificationPipeline
from transformers import AutoTokenizer, AutoModelForSequenceClassification
model = AutoModelForSequenceClassification.from_pretrained("pritamdeka/PubMedBert-PubMed200kRCT")
tokenizer = AutoTokenizer.from_pretrained("pritamdeka/PubMedBert-PubMed200kRCT")
pipe = TextClassificationPipeline(model=model, tokenizer=tokenizer, return_all_scores=True)
pipe("Treatment of 12 healthy female subjects with CDCA for 2 days resulted in increased BAT activity.")
Results will be shown as follows:
[[{'label': 'BACKGROUND', 'score': 0.0028450002428144217},
{'label': 'CONCLUSIONS', 'score': 0.2581048607826233},
{'label': 'METHODS', 'score': 0.015086210332810879},
{'label': 'OBJECTIVE', 'score': 0.0016815993003547192},
{'label': 'RESULTS', 'score': 0.7222822904586792}]]
More information needed
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 0.3604 | 0.14 | 5000 | 0.3162 | 0.8821 |
| 0.3326 | 0.29 | 10000 | 0.3112 | 0.8843 |
| 0.3293 | 0.43 | 15000 | 0.3044 | 0.8870 |
| 0.3246 | 0.58 | 20000 | 0.3040 | 0.8871 |
| 0.32 | 0.72 | 25000 | 0.2969 | 0.8888 |
| 0.3143 | 0.87 | 30000 | 0.2929 | 0.8903 |
| 0.3095 | 1.01 | 35000 | 0.2917 | 0.8899 |
| 0.2844 | 1.16 | 40000 | 0.2957 | 0.8886 |
| 0.2778 | 1.3 | 45000 | 0.2943 | 0.8906 |
| 0.2779 | 1.45 | 50000 | 0.2890 | 0.8935 |
| 0.2752 | 1.59 | 55000 | 0.2881 | 0.8919 |
| 0.2736 | 1.74 | 60000 | 0.2835 | 0.8944 |
| 0.2725 | 1.88 | 65000 | 0.2833 | 0.8942 |
pytorch_model.bin 438 MBmodel.safetensors 438 MBtraining_args.bin 3 kBmicrosoft/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext