fin-lora-news-qwen2.5-1.5b

Standard adapter (no adversarial training): 88.3% clean accuracy on 2,368 test tweets, but 50.6% after one planted sentence; for untrusted text use fin-lora-news-qwen2.5-1.5b-adv (see Re-evaluation)

A LoRA adapter for Qwen/Qwen2.5-1.5B-Instruct that classifies short English financial text (news snippets, headlines, tweets) as Bearish, Bullish or Neutral.

The base model stays a causal language model. The label is read from the next-token logits of three label tokens at the end of a fixed prompt; no text is generated. Label ids: 0 Bearish, 1 Bullish, 2 Neutral.

• Author: Nagizaaz Shaik
• Code, result files and the full write-up: https://github.com/shaikn6/fin-lora
• Adapter weights: 18,464,768 parameters (counted from adapter_model.safetensors)

Intended use

Research and education: studying how small fine-tuned language models classify financial sentiment, how they compare with cheaper baselines, and how easily they are manipulated by text added to the input.

Out-of-scope use

• This is not investment advice, and the output is not a prediction of price movement.
• Do not use it for automated trading decisions or any decision that moves money. Appending one sentence to the input changes the label on a large share of items (see Robustness).
• It has not been evaluated on filings, earnings-call transcripts, non-English text or inputs longer than the 256-token window.

Training data

11,512 examples (train_news-1.5b.json, examples), defined once in finlora/news.py (train_mix):

Licence notes:

• None of the three training datasets carries a non-commercial or no-derivatives term on its card (checked 2026-10-10). ODC-By requires attribution; the NOSIBLE dataset is credited above.
• The NOSIBLE labels were produced by language models from several vendors and its text is collected from public news sites. Anyone planning commercial use should review that themselves; the adapter licence below does not settle it.
• The robustness figures on this card were measured on other datasets that are restricted: Financial PhraseBank (CC-BY-NC-SA-3.0), FOMC communication (CC-BY-NC-4.0) and a gold-news set (CC-BY-NC-ND-4.0), plus an oil-headline set (CC-BY-4.0) and a crypto set (MIT per card metadata). Those datasets were used for inference only. The adapter was not trained on them, no text from them is distributed here, and only aggregate rates are reported. Licences as recorded in markets_manifest.json in the repository.

The adapter is released under Apache-2.0, the licence of the base model.

Training procedure

• Command: python train_news.py Qwen/Qwen2.5-1.5B-Instruct news-1.5b 8000 2000 256 8 2
• LoRA: r 16, alpha 32, dropout 0.05, no bias, target modules down_proj, gate_proj, k_proj, o_proj, q_proj, up_proj, v_proj (adapter_config.json).
• Objective: cross-entropy over the three label-token logits at the last prompt position.
• Optimiser: AdamW, peak learning rate 2e-4, weight decay 0.01, one-cycle schedule with 5% warm-up, gradient clipping at 1.0, seed 0 (train_news.py).
• Batch size 8, gradient accumulation 2 (from the command above; not stored in the run summary).
• 719 optimiser steps, one pass over the data, token window 256 (train_news-1.5b.json: steps, max_len).
• Base weights loaded in bfloat16; gradient checkpointing on.
• Training time: 11,996 s (about 3.3 h) (train_news-1.5b.json, train_seconds).
• Hardware: a laptop GPU, per the repository README. The run summary does not record the hardware.
• Library versions pinned in the repository: peft 0.21.1, transformers 5.18.0, torch 2.14.1.

Evaluation results

This adapter was not run through the three-set evaluation script (eval_news.py), so there are no committed tweet or news-snippet test numbers for it, and no macro-F1 on headlines. The dev split was scored at the end of training; it was not used to choose hyperparameters or checkpoints.

How to read these numbers:

• The NOSIBLE labels are machine-made and the adapter trained on the same kind of labels, so accuracy on NOSIBLE splits overstates agreement with human judgement.
• The headline set is the only one with manually validated labels. It has 267 items, so its margin of error is about 5 points.
• One training run with one seed. There are no confidence intervals on these accuracies.

Robustness: the model can be steered by planted text

A sentence naming a label was appended to each input, and the model's answer was compared with its answer on the unedited text. The metric is the forced rate as defined in the repository README: of the items the model did not already give the planted label, the share pushed to it (JSON field forced_rate_excl_already_target).

Each cell is the lowest and highest value over five markets (equities, oil, gold, central-bank statements, crypto), 300 items per market, sampling seed 0. Sources: robustness_markets_lora_news_1.5b.json (instruction row) and robustness_markets_variants_lora_news_1.5b.json (other rows), model lora_news_1.5b.

Plainly: this model is vulnerable to planted text. The most effective phrasing here, "Annotation" with X = Neutral, moved 86.3-98.0% of the items that did not already have that label. On the 267 manually labeled headlines the instruction phrasing gives: Bearish 3.4%, Bullish 19.3%, Neutral 95.7% (robustness_equities_lora_news_1.5b.json).

The instruction rows were repeated with two more sampling seeds (robustness_markets_lora_news_1.5b_seed1.json, robustness_markets_lora_news_1.5b_seed2.json). For the Neutral instruction the ranges are seed 1: 42.7-83.8%; seed 2: 47.5-85.3%. The other three phrasings were run with seed 0 only.

These are label-change rates. Nothing here measures trading losses or any real-world harm.

Re-evaluation (2026-10-28)

This adapter (unchanged) was re-scored with the evaluation used for fin-lora-news-qwen2.5-1.5b-adv: test items that also appear in the training data (after lower-casing, stripping punctuation and links) removed, bfloat16, the text truncated instead of the prompt, 95% bootstrap intervals over items. "After planting" appends one of the demo Space's four sentences naming a wrong label (both wrong labels; tweets: a seeded 1,000 items).

A retrained candidate for this repo (all tweets, two epochs, no planted augmentation) raised tweets macro-F1 from 0.858 to 0.887 (paired change +2.9 points, 95% CI [1.7, 4.1]), but its accuracy after planting fell on news snippets (58.8% to 52.9%, change CI [-7.8, -4.1]), so it was not published. The adversarially trained 1.5B adapter keeps clean accuracy (tweets macro-F1 0.884) and holds 91.1% accuracy after planting on tweets.

Limitations

• Vulnerable to planted label text, as shown above. Do not feed it untrusted text and act on the output.
• Small test set with human labels (267 headlines, 12 of them negative). Results on negative headlines are not reliable.
• Most training labels are machine-made (NOSIBLE).
• English only. Short text only: prompts are truncated at 256 tokens from the right, which removes the final Sentiment: cue on very long inputs. Trim the text, not the prompt, if your inputs are long.
• One base model size, one training seed, one pass over the data.
• The three classes are a coarse view of sentiment and say nothing about magnitude or time horizon.

How to use

The prompt template and token window ship with the adapter in prompt.json and must be reproduced exactly.

import json
from pathlib import Path

import torch
from huggingface_hub import snapshot_download
from peft import PeftConfig, PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

ADAPTER = "fin-lora-news-qwen2.5-1.5b"  # a local folder, or "9mark9/fin-lora-news-qwen2.5-1.5b" once published
LABELS = ["Bearish", "Bullish", "Neutral"]

adapter_dir = Path(ADAPTER) if Path(ADAPTER).is_dir() else Path(snapshot_download(ADAPTER))
prompt_cfg = json.loads((adapter_dir / "prompt.json").read_text())
base_id = PeftConfig.from_pretrained(str(adapter_dir)).base_model_name_or_path  # Qwen/Qwen2.5-1.5B-Instruct

device = "mps" if torch.backends.mps.is_available() else "cuda" if torch.cuda.is_available() else "cpu"
tok = AutoTokenizer.from_pretrained(base_id)
tok.padding_side = "left"
base = AutoModelForCausalLM.from_pretrained(base_id, dtype=torch.bfloat16)  # older transformers: torch_dtype=
model = PeftModel.from_pretrained(base, str(adapter_dir)).eval().to(device)

label_ids = [tok(" " + label, add_special_tokens=False).input_ids[0] for label in LABELS]


@torch.no_grad()
def classify(texts):
    prompts = [prompt_cfg["template"].format(text=t) for t in texts]
    enc = tok(prompts, return_tensors="pt", padding=True, truncation=True, max_length=prompt_cfg["max_len"]).to(device)
    logits = model(**enc).logits[:, -1, :][:, label_ids]
    probs = torch.softmax(logits.float(), dim=-1).cpu()
    return [(LABELS[int(p.argmax())], [round(float(x), 3) for x in p]) for p in probs]


print(classify(["Shares fell 12% after the company cut its full-year revenue forecast."]))

This mirrors the repository's own loader (finlora/arena.py, finlora/common.py): the tokenizer comes from the base model, padding is on the left, and the class is the argmax over the first tokens of " Bearish", " Bullish" and " Neutral".

Repository

https://github.com/shaikn6/fin-lora