FinTwitRoBERTa-sentiment

FinTwitRoBERTa-sentiment is a fine-tuned model for classifying the sentiment of financial tweets as bullish, bearish or neutral. It uses FinTwitRoBERTa-large as a base model, a RoBERTa-large model (355M parameters) pre-trained on millions of financial tweets. This approach ensures that FinTwitRoBERTa-sentiment has seen enough financial tweets, which have an informal nature, compared to other financial texts, such as news headlines. Therefore this model performs great on informal financial texts, seen on social media. It is the successor of FinTwitBERT-sentiment.

Intended Uses

FinTwitRoBERTa-sentiment is intended for classifying financial tweets or other financial social media texts, such as a feed of tweets about stocks and crypto. A tweet is bullish or bearish only when its author expresses an expectation or position; facts, news, questions and price alerts are neutral, which is most of a typical feed. The sentiment is that of the tweet's own text: for a quote tweet, classify the new text without the quoted tweet.

Dataset

FinTwitRoBERTa-sentiment has been trained on human-labeled financial tweets from two datasets:

• TimKoornstra/financial-tweets-sentiment: 24,373 human-labeled tweets after cleaning. The sources without a neutral class or with inconsistent labels were left out, and the bullish and bearish tweets of the other sources were relabeled by hand with one labeling guideline (most became neutral, since they state facts or news rather than an opinion).
• 1,500 tweets from StephanAkkerman/financial-tweets, posted from 2024 onwards, picked where an earlier model was least sure and labeled with the same guideline (data/train/feed_v1.csv)

In total 23,435 tweets for training and 2,438 for validation. The tweets of our dev and test sets were removed from the training data.

Model Details

• Base model: FinTwitRoBERTa-large, a RoBERTa-large model (355M parameters) pre-trained on millions of financial tweets.
• Labels: NEUTRAL, BULLISH, BEARISH.
• Training: 2 epochs, learning rate 2e-05 with 10% warmup and layer-wise learning-rate decay of 0.9, weight decay 0.01, batch size 32, maximum length 128 tokens, seed 42. The checkpoint with the best validation F1 (2,438 tweets held out from the training sources) is kept. The recipe (llrd-0.9-feed in experiments.json) was chosen out of a grid of learning rates, epochs, class weights, label smoothing and layer-wise decay by its macro F1 on the dev set.
• Calibration: config.calibration_temperature = 1.046, fitted on the dev set. Divide the logits by it before the softmax to get confidence scores that match how often the model is right.

Evaluation

The test set has 1,237 tweets from a real financial Twitter feed (first seen from August 2023 onwards) and the dev set 749 tweets from the same feed (2024 onwards), labeled with the same guideline. Neither was used for training; the dev set was used to choose the recipe and fit the temperature. The 95% bootstrap interval of the test macro F1 is 0.871–0.910. For comparison, FinTwitBERT-sentiment scores a macro F1 of 0.565 on the same test set.

More Information

For a comprehensive overview, including the training setup and analysis of the model, visit the ModernFinTwitBERT GitHub repository.

Usage

Using HuggingFace's transformers library the model and tokenizers can be converted into a pipeline for text classification.

from transformers import pipeline

# Create a sentiment analysis pipeline
pipe = pipeline(
    "sentiment-analysis",
    model="StephanAkkerman/FinTwitRoBERTa-sentiment",
)

# Get the predicted sentiment
print(pipe("Nice 9% pre market move for $para, pump my calls Uncle Buffett 🤑"))

The model was trained on tweets with links replaced by [URL] and user mentions by @USER, so do the same for the best results. The pipeline's scores are not calibrated; for confidence scores, divide the logits by config.calibration_temperature:

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("StephanAkkerman/FinTwitRoBERTa-sentiment")
model = AutoModelForSequenceClassification.from_pretrained("StephanAkkerman/FinTwitRoBERTa-sentiment").eval()

inputs = tokenizer(["$NVDA breaking out, loading calls"], return_tensors="pt", truncation=True)
with torch.no_grad():
    logits = model(**inputs).logits
probs = (logits / model.config.calibration_temperature).softmax(-1)
print({model.config.id2label[i]: p for i, p in enumerate(probs[0].tolist())})

Decision thresholds

config.decision_thresholds = {"bullish": 0.8, "bearish": 0.86}, tuned on the dev set for precision >= 0.9. Call a tweet bullish or bearish only when its calibrated probability for that class reaches the threshold, otherwise neutral:

label = model.config.id2label[int(probs[0].argmax())].lower()
if label != "neutral" and probs[0].max() < model.config.decision_thresholds[label]:
    label = "neutral"

ONNX int8

onnx/model_int8.onnx is this model with int8 weights (dynamic per-channel quantization of all linear layers and the embeddings) for fast CPU inference, for example on a Raspberry Pi. It needs only onnxruntime and tokenizers, no torch:

import json

import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download
from tokenizers import Tokenizer

repo = "StephanAkkerman/FinTwitRoBERTa-sentiment"
config = json.loads(open(hf_hub_download(repo, "config.json")).read())
session = ort.InferenceSession(hf_hub_download(repo, "onnx/model_int8.onnx"))
tokenizer = Tokenizer.from_file(hf_hub_download(repo, "tokenizer.json"))
tokenizer.enable_truncation(128)
tokenizer.enable_padding(pad_id=1)

encodings = tokenizer.encode_batch(["$NVDA breaking out, loading calls"])
logits = session.run(None, {
    "input_ids": np.array([e.ids for e in encodings], dtype=np.int64),
    "attention_mask": np.array([e.attention_mask for e in encodings], dtype=np.int64),
})[0] / config["calibration_temperature"]
probs = np.exp(logits) / np.exp(logits).sum(-1, keepdims=True)  # NEUTRAL, BULLISH, BEARISH

Apply config.decision_thresholds to probs the same way as for the PyTorch model. ModernFinTwitBERT's onnx_predict.OnnxSentiment does all of this, including the training preprocessing (@USER, [URL]). Scores on the same dev and test sets; same_label_as_fp32 is how often both models give the same argmax label:

Citing & Authors

If you use FinTwitRoBERTa-sentiment or FinTwitRoBERTa-large in your research, please cite me as follows:

@misc{FinTwitRoBERTalarge,
  author = {Stephan Akkerman},
  title = {FinTwitRoBERTa-large},
  year = {2026},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/StephanAkkerman/ModernFinTwitBERT}}
}

Additionally, if you utilize the sentiment classifier, please cite:

@misc{FinTwitRoBERTasentiment,
  author = {Stephan Akkerman},
  title = {FinTwitRoBERTa-sentiment: A Sentiment Classifier for Financial Tweets},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/StephanAkkerman/FinTwitRoBERTa-sentiment}}
}

License

This project is licensed under the MIT License. See the LICENSE file for details.