Fine-Tuned Topic Classification Model - SK_BPE_BLM (Topic News)

Model Overview

This model is a fine-tuned version of the SK_BPE_BLM model for topic classification. For this task, we used the Slovak Categorized News Corpus, which contains news articles divided into six categories: Economy and Business, Culture, News, World, Sports, and Healthcare. The corpus provides text files with detailed annotations, including token and sentence boundary identification, stop words, morphological analysis, named entity recognition, and lemmatization.

Topic Labels

Each record in the dataset is labeled with one of the following topics:

• Healthcare (0): 2,564 records
• News (1): 4,174 records
• Sports (2): 2,759 records
• World (3): 1,660 records
• Economy and Business (4): 4,199 records
• Culture (5): 137 records

Dataset Details

The original corpus did not contain continuous text, requiring significant preprocessing. The process involved:

1. Reconstruction: We reconstructed coherent text from individual annotated files, resulting in over 86,000 sentences.
2. Combining Sentences: Sentences from each file were combined into single records, with a maximum length of 600 characters (approximately 200 tokens). The final dataset comprises a total of 15,493 records, each labeled according to the categories listed above.

For more information about the dataset, please visit this link.

Fine-Tuning Hyperparameters

The following hyperparameters were used during the fine-tuning process:

• Learning Rate: 1e-05
• Training Batch Size: 64
• Evaluation Batch Size: 64
• Seed: 42
• Optimizer: Adam (default)
• Number of Epochs: 10

Model Performance

The model was evaluated using stratified 10-fold cross-validation, achieving a weighted F1-score with a median value of 0.966 across the folds.

Model Usage

This model is suitable for topic classification in Slovak text, particularly for news articles across various categories. It is specifically designed for applications requiring topic categorization of news content and may not generalize well to other types of text.

Example Usage

Below is an example of how to use the fine-tuned SK_Morph_BLM-topic-news model in a Python script:

import torch
from transformers import RobertaForSequenceClassification, RobertaTokenizerFast

class TopicClassifier:
    def __init__(self, tokenizer, model):
        self.model = RobertaForSequenceClassification.from_pretrained(model, num_labels=6)
        self.tokenizer = RobertaTokenizerFast.from_pretrained(tokenizer, max_length=256)

    def tokenize_text(self, text):
        encoded_text = self.tokenizer.encode_plus(
            text.lower(), 
            max_length=256, 
            padding='max_length', 
            truncation=True, 
            return_tensors='pt'
        )
        return encoded_text

    def classify_text(self, encoded_text):
        with torch.no_grad():
            output = self.model(**encoded_text)
            logits = output.logits
            predicted_class = torch.argmax(logits, dim=1).item()
            probabilities = torch.softmax(logits, dim=1)
            class_probabilities = probabilities[0].tolist()
            predicted_class_text = self.model.config.id2label[predicted_class]
        return predicted_class, predicted_class_text, class_probabilities

# Instantiate the topic classifier with the specified tokenizer and model
classifier = TopicClassifier(tokenizer="daviddrzik/SK_BPE_BLM", model="daviddrzik/SK_BPE_BLM-topic-news")

# Example text to classify topic
text_to_classify = "Tento dôležitý zápas medzi Českou republikou a Švajčiarskom sa po troch tretinách skončil 2:0."
print("Text to classify: " + text_to_classify + "\n")

# Tokenize the input text
encoded_text = classifier.tokenize_text(text_to_classify)

# Classify the topic of the tokenized text
predicted_class, predicted_class_text, logits = classifier.classify_text(encoded_text)

# Print the predicted class label and index
print(f"Predicted class: {predicted_class_text} ({predicted_class})")
# Print the probabilities for each class
print(f"Class probabilities: {logits}")

Example Output Here is the output when running the above example:

Text to classify: Tento dôležitý zápas medzi Českou republikou a Švajčiarskom sa po troch tretinách skončil 2:0.

Predicted class: Sport (2)
Class probabilities: [0.0005132766091264784, 0.0001338021393166855, 0.9980706572532654, 0.00035721372114494443, 0.0005179148865863681, 0.00040712326881475747]