KenyaESG-RoBERTa-gov (v2)

Binary sentence classifier that flags governance ESG content in corporate reports of firms listed on the Nairobi Securities Exchange (NSE). It is one of three independent pillar classifiers (env, soc, gov), fine-tuned from roberta-base on the KenyaESG dataset.

• Output: 1 if the sentence addresses governance matters, 0 otherwise (threshold 0.5 on the class-1 probability). Aggregated over a report, the share of positive sentences gives a firm-year governance disclosure-intensity score.
• Scope of the pillar: board composition, meetings, committees and appointments, remuneration, audit and internal control, risk management, compliance, shareholder rights and AGMs, ethics and anti-corruption.

What changed in v2 (November 2026)

The model was retrained on revised training labels: all Kenyan training sentences were re-read against an explicit, meaning-based annotation guideline (124 labels changed for this pillar), and it is now evaluated on an extended 300-sentence evaluation set with an adjudicated reference label. The first-release model (DOI 10.57967/hf/9128) reported F1 0.916 on a 100-sentence benchmark drawn from the same pool as the training data; that benchmark overstated performance on the wider corpus (a model trained with the same procedure on the first-release labels scores F1 0.791 on the 300-sentence set against the adjudicated label). See the dataset card for details.

Usage

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

repo = "josephagossa/KenyaESG-RoBERTa-gov"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo).eval()

sentences = ["The Board held eight meetings during the year.",
             "Revenue grew by 12% compared with the prior year."]
enc = tok(sentences, truncation=True, max_length=256, padding=True, return_tensors="pt")
with torch.no_grad():
    prob = torch.softmax(model(**enc).logits, dim=-1)[:, 1]
labels = (prob >= 0.5).int().tolist()   # 1 = governance content

Evaluation

300 held-out sentences from NSE reports, labelled by two annotators (A, B); scored against the adjudicated label (headline) and, as an independent check, against the sentences on which A and B agree. 95% percentile bootstrap intervals (10,000 resamples).

The training labels and the adjudicated evaluation labels were revised with the same guideline, so the adjudicated score may be slightly favourable; the A = B score does not depend on the adjudication.

Internal validation (585 sentences, measures agreement with the training labels): F1 0.888 overall, 0.969 on Kenyan sentences, 0.718 on Schimanski et al. (2024) sentences.

Training

• Data: 3,900 sentences — Kenyan reports 1,068 positive / 865 negative; Schimanski et al. (2024) 526 / 1,441. Split 85/15 (3,315 / 585), stratified on label × source.
• Labels: Kenyan sentences: keyword filter, reviewed by the author, then re-read against the KenyaESG guideline v2. Schimanski et al. sentences: labels as released by the original authors.
• Hyperparameters: learning rate 2e-5, batch size 16, weight decay 0.01, up to 4 epochs with early stopping (patience 2, best validation F1), max length 256, fp16, seed 42.
• Software / hardware: Python 3.12.7, PyTorch 2.5.1 (CUDA 12.1), Transformers 4.48.1, Datasets 3.3.2, scikit-learn 1.5.1; NVIDIA Quadro RTX 4000.

Limitations

• Measures how often the pillar is mentioned (disclosure intensity), not the quality or credibility of disclosure.
• Precision is high (0.95); recall is lower against the annotators' own labels (0.78 on sentences where both agree), so the model remains somewhat conservative. Validation F1 on Schimanski et al. sentences is only 0.72, as their governance definition is narrower than the KenyaESG guideline.
• Trained on predominantly English NSE reports (a few Swahili and OCR-degraded sentences); transfer to other markets or languages is untested.
• Company and person names in the training data are masked as [COMPANY] / [PERSON]; apply the same masking for best results.

Citation

Agossa, J. (2026). Pricing the Cost of Compliance: Equity Reactions to Mandatory ESG Disclosure in Kenya. Working paper. SSRN.

Dataset: KenyaESG, Zenodo, https://doi.org/10.5281/zenodo.20608236 · Hugging Face, https://doi.org/10.57967/hf/9193

Labelling schema adapted from: Schimanski, T., Reding, A., Reding, N., Bingler, J., Kraus, M., & Leippold, M. (2024). Bridging the gap in ESG measurement: Using NLP to quantify environmental, social, and governance communication. Finance Research Letters, 61, 104979.

License: Apache-2.0.