GIST-small-finefineweb

A fine-tuned version of the bert architecture (BertForSequenceClassification) optimized for the text-classification task.

• Model type: bert
• Problem Type: single_label_classification
• Number of Labels: 67
• Vocabulary Size: 30522
• License: MIT

Use

To get started with this model in Python using the Hugging Face Transformers library, run the following code:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "agentlans/GIST-small-finefineweb"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "Replace this with your input text."
inputs = tokenizer(text, return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predicted_class_id = logits.argmax().item()
predicted_class_name = model.config.id2label[predicted_class_id]

print(f"Predicted Class ID: {predicted_class_id}")
print(f"Predicted Class Name: {predicted_class_name}")

Intended Uses & Limitations

Intended Use

This model is designed for sequence classification tasks. Below are the specific class labels mapped to their corresponding IDs:

Training Details

Hyperparameters

The following hyperparameters were used during fine-tuning:

• Learning Rate: 5e-05
• Train Batch Size: 8
• Eval Batch Size: 8
• Optimizer: OptimizerNames.ADAMW_TORCH_FUSED
• Number of Epochs: 3.0
• Mixed Precision: BF16

Show Advanced Training Configuration

Optimization & Regularization

• Gradient Accumulation Steps: 1
• Learning Rate Scheduler: SchedulerType.LINEAR
• Warmup Steps: 0
• Warmup Ratio: None
• Weight Decay: 0.0
• Max Gradient Norm: 1.0

Hardware & Reproducibility

• Number of GPUs: 1
• Seed: 42

Training Results & Evaluation

During fine-tuning, the model achieved the following results on the evaluation set:

For performance on the test set, click here.

Speed Performance

• Training Runtime: 720.633 seconds
• Train Samples per Second: 474.166
• Evaluation Runtime: 10.5434 seconds
• Eval Samples per Second: 1906.406

Show Detailed Training Logs

Training Logs History

Framework Versions

• Transformers: 5.14.0.dev0
• PyTorch: 2.9.1+cu128