📙 SPECTER2–WoS (Multiclass Classification on Web of Science Research Areas)

This model is a fine-tuned version of allenai/specter2_base for multiclass bibliometric classification using Web of Science (WoS) Research Areas.

It achieves the following results on the evaluation set:

• Loss: 1.8987
• Accuracy: 0.5763
• Precision Micro: 0.5763
• Precision Macro: 0.5744
• Recall Micro: 0.5763
• Recall Macro: 0.5741
• F1 Micro: 0.5763
• F1 Macro: 0.5691

Model description

This model fine-tunes SPECTER2 (allenai/specter2_base) to classify scientific publications into Web of Science (WoS) Research Areas, one of the most widely used bibliometric taxonomies.

WoS Research Areas represent a 153 broad-field disciplinary classification assigned at the journal level. The model takes title, abstract, or title + abstract as input and assigns exactly one WoS Research Area using a softmax classifier.

Key characteristics:

• Base model: allenai/specter2_base
• Task: multiclass document classification
• Labels: ~150 WoS Research Areas
• Activation: softmax
• Loss: CrossEntropyLoss
• Output: single best-matching Research Area

This model provides high-level

Intended uses & limitations

Intended uses

This model is designed for:

• Assigning publications to Web of Science Research Areas
• Enriching or correcting metadata in:
  • institutional repositories
  • research information systems
  • grant and project databases
  • bibliometric monitoring tools
• Supporting scientometric tasks:
  • global discipline mapping
  • portfolio profiling
  • disciplinary trend analysis
• Classifying scientific publications when only title/abstract is available

Works with:

• title only
• abstract only
• title + abstract (recommended)

Limitations

• WoS Research Areas are journal-level labels, not article-level annotations.
  → This introduces noise, especially for multidisciplinary journals.
• Some Research Areas have low representation, affecting macro-F1.
• Multiclass classification forces each article into one label, even if it spans multiple disciplines.
• Not suitable for:
  • fine-grained WoS Categories (SCs)
  • ASJC Areas or MAG FoS models (use separate classifiers)
  • normative evaluation or ranking of research
  • decisions requiring verified human annotations

Predictions should be interpreted as high-level field approximations, not exact domain attribution.

Training and evaluation data

Training data consists of documents with Web of Science Research Area labels, derived from journal-level classification.
Because WoS SC/RAs are not publicly distributed, we rely on a curated dataset constructed from open-access publications where Research Areas can be inferred from the journal metadata.

Notes on WoS Research Areas

• Research Areas are high-level and non-hierarchical.
• Articles inherit the journal’s assigned Research Area(s).
• Some journals have multiple RAs, but this model uses a single predominant label for training.

Training procedure

Preprocessing

• Input text constructed as:
  title + ". " + abstract
• Tokenization using SPECTER2 tokenizer
• Maximum sequence length: 512 tokens

Model

• Base model: allenai/specter2_base
• Classification head: linear → softmax
• Loss: CrossEntropyLoss

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Evaluation results

Framework versions

• Transformers 4.57.1
• Pytorch 2.8.0+cu126
• Datasets 3.6.0
• Tokenizers 0.22.1