CEO responsible-leadership classifier (ModernBERT-base)

Binary text classifier that decides whether a sentence from a CEO letter to shareholders expresses responsible leadership, an orientation towards stakeholders, employees, society, the environment, ethics or long-term value, rather than ordinary business or financial reporting. Fine-tuned from answerdotai/ModernBERT-base on 5,418 sentences labeled by human coders. Built for the analyses in Kirsch, Gijselaers, Lokshin and Vanstraelen (2026), The Leadership Quarterly; see the citation at the end. English only. The input is a single sentence and nothing else.

Labels

The model returns a probability for responsible; the argmax decision (threshold 0.50) is the one used in all reported results. Human coders labeled 71% of the sentences responsible, and the model reproduces that share (70.7% on the full sample).

Usage

from transformers import pipeline

clf = pipeline(
    "text-classification",
    model="LRieser/ceo-responsible-leadership-modernbert-base",
    truncation=True,
    max_length=256,
)
clf("We are committed to creating long-term value for all our stakeholders.")
# [{'label': 'responsible', 'score': 0.99}]

Batched scoring (about 700 sentences per second on an RTX 3090 at batch 64):

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

repo = "LRieser/ceo-responsible-leadership-modernbert-base"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo).eval().cuda()

@torch.inference_mode()
def predict(texts, batch_size=64):
    out = []
    for i in range(0, len(texts), batch_size):
        enc = tok(texts[i:i + batch_size], padding=True, truncation=True,
                  max_length=256, return_tensors="pt").to(model.device)
        out.extend(model(**enc).logits.argmax(-1).tolist())
    return out

Pass one sentence at a time, split from the letter beforehand. The model was trained on single sentences; longer passages were never seen in training and are truncated at 256 tokens.

Training

• Base model: answerdotai/ModernBERT-base (22 layers, about 149M parameters), sequence-classification head with two labels.
• Data: 5,418 sentences randomly sampled from CEO letters to shareholders of listed companies and labeled by PhD-level human coders from the authors' team: 3,869 responsible (71%) and 1,549 not_responsible. Sentences were sampled and coded without reference to the keyword dictionary from the paper, so keyword presence gives no structural advantage. The labeled data are not distributed with the model.
• Protocol: repeated stratified 5-fold cross-validation with 3 repeats (15 runs, seed 42). Each run trains on about 4,334 sentences and scores its held-out fold of about 1,084.
• Loss: class-weighted cross-entropy (balanced weights, 1.75 on not_responsible and 0.70 on responsible).
• Optimisation: AdamW, learning rate 2e-5, linear schedule, warmup ratio 0.1, weight decay 0.01, gradient clipping 1.0, batch size 16, maximum sequence length 256, fp16 mixed precision.
• Schedule: 3 epochs per run, one evaluation per epoch on the held-out fold; each run keeps its best epoch by macro-F1. The released checkpoint is the run with the highest held-out macro-F1 (0.916, accuracy 0.932). That figure is the maximum over 15 runs; the cross-validated figures below are the expected performance on new sentences.

Evaluation

Mean and standard deviation over the 15 cross-validation runs, each scored on its own held-out fold of about 1,084 sentences. Every sentence is scored out-of-fold three times, once per repeat. The 95% confidence interval over runs is 0.883 to 0.897 for macro-F1 and 0.952 to 0.960 for ROC-AUC.

Pooled confusion matrix over the 16,254 out-of-fold predictions (rows: true, columns: predicted):

Most false positives are values statements and aspirational boilerplate ("core values: safety, environmental stewardship, highest ethical behaviour") that the coders judged as lacking a substantive commitment. Most false negatives are statements about financial strength that the coders linked to stakeholder value creation.

The keyword dictionary from the paper (64 terms, regular-expression matching with wildcards) reaches accuracy 0.695 and responsible-class F1 0.804 against the same human labels; the classifier and the dictionary agree on 69% of sentences. Full metrics, including all 15 runs, are in eval_results.json.

Scoring the 5,418 sentences takes about 8 seconds on an RTX 3090.

Citation

@misc{rieser2026ceoresponsibleleadershipmodernbert,
  author    = {Rieser, Lars and Kirsch, Laura and Gijselaers, Wim and Lokshin, Boris and Vanstraelen, Ann},
  title     = {CEO responsible-leadership classifier (ModernBERT-base)},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/LRieser/ceo-responsible-leadership-modernbert-base}
}

The model was built for the following paper, which describes the construct, the coding and the keyword dictionary. Please cite it as well:

@article{kirsch2026measuring,
  author  = {Kirsch, Laura and Gijselaers, Wim and Lokshin, Boris and Vanstraelen, Ann},
  title   = {Measuring {CEO} responsible leadership: Development and validation of a linguistic-based instrument},
  journal = {The Leadership Quarterly},
  volume  = {37},
  number  = {3},
  pages   = {101963},
  year    = {2026},
  doi     = {10.1016/j.leaqua.2026.101963}
}