📗 SPECTER2–MAG (Multiclass Classification on MAG Level-0 Fields of Study)

This model is a fine-tuned version of allenai/specter2_base for multiclass bibliometric classification using MAG Fields of Study – Level 0 (SciDocs). It achieves the following results on the evaluation set:

• Loss: 1.0598
• Accuracy: 0.8310
• Precision Micro: 0.8310
• Precision Macro: 0.8290
• Recall Micro: 0.8310
• Recall Macro: 0.8276
• F1 Micro: 0.8310
• F1 Macro: 0.8263

Model description

This model is a fine-tuned version of SPECTER2 (allenai/specter2_base) adapted for multiclass classification across the 19 top-level Fields of Study (FoS) from the Microsoft Academic Graph (MAG).

The model accepts the title, abstract, or title + abstract of a scientific publication and assigns it to exactly one of the MAG Level-0 domains (e.g., Biology, Chemistry, Computer Science, Engineering, Psychology).

Key characteristics:

• Base model: allenai/specter2_base
• Task: multiclass document classification
• Labels: 19 MAG Field of Study Level-0 categories
• Activation: softmax
• Loss: CrossEntropyLoss
• Output: single best-matching FoS category

MAG Level-0 represents broad disciplinary domains designed for high-level categorization of scientific documents.

Intended uses & limitations

Intended uses

This multiclass MAG model is suitable for:

• Assigning publications to top-level scientific disciplines
• Enriching metadata in:
  • repositories
  • research output systems
  • funding and project datasets
  • bibliometric dashboards
• Supporting scientometric analyses such as:
  • broad-discipline portfolio mapping
  • domain-level clustering
  • modeling research diversification
• Classifying documents when only title/abstract is available

The model supports inputs such as:

• title only
• abstract only
• title + abstract (recommended)

Limitations

• MAG Level-0 categories are very coarse (e.g., Biology, Medicine, Engineering), and do not represent subfields.
• Documents spanning multiple fields must be forced into one label—an inherent limitation of multiclass classification.
• The training labels come from MAG’s automatic field assignment pipeline, not manual expert annotation.
• Not suitable for:
  • fine-grained subdisciplines
  • downstream tasks requiring multilabel outputs
  • WoS Categories or ASJC Areas (use separate models)
  • clinical or regulatory decision-making

Predictions should be treated as high-level disciplinary metadata, not detailed field classification.

Training and evaluation data

Source dataset: SciDocs

Training data comes from the SciDocs dataset, introduced together with the original SPECTER paper:

SciDocs provides citation graphs, titles, abstracts, and MAG Fields of Study for scientific documents derived from MAG.
For this model, we use MAG Level-0 FoS, the 19 top-level scientific domains.

Dataset characteristics:

Training procedure

Preprocessing

• Input text constructed as:
  abstract
• Tokenization using the SPECTER2 tokenizer
• Maximum sequence length: 512 tokens

Model

• Base model: allenai/specter2_base
• Classification head: linear layer → softmax
• Loss: CrossEntropyLoss

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Evaluation results

Framework versions

• Transformers 4.57.1
• Pytorch 2.8.0+cu126
• Datasets 3.6.0
• Tokenizers 0.22.1