fin-lora-news-qwen2.5-0.5b

Standard adapter (no adversarial training): 86.7% clean accuracy on 2,368 test tweets, but 64.1% after one planted sentence; for untrusted text use fin-lora-news-qwen2.5-0.5b-adv (see Re-evaluation)

A LoRA adapter for Qwen/Qwen2.5-0.5B-Instruct that classifies short English financial text (news snippets, headlines, tweets) as Bearish, Bullish or Neutral.

The base model stays a causal language model. The label is read from the next-token logits of three label tokens at the end of a fixed prompt; no text is generated. Label ids: 0 Bearish, 1 Bullish, 2 Neutral.

• Author: Nagizaaz Shaik
• Code, result files and the full write-up: https://github.com/shaikn6/fin-lora
• Adapter weights: 8,798,208 parameters (counted from adapter_model.safetensors)

Intended use

Research and education: studying how small fine-tuned language models classify financial sentiment, how they compare with cheaper baselines, and how easily they are manipulated by text added to the input.

Out-of-scope use

• This is not investment advice, and the output is not a prediction of price movement.
• Do not use it for automated trading decisions or any decision that moves money. Appending one sentence to the input changes the label on a large share of items (see Robustness).
• It has not been evaluated on filings, earnings-call transcripts, non-English text or inputs longer than the 256-token window.

Training data

11,512 examples (train_news-0.5b.json, examples), defined once in finlora/news.py (train_mix):

Licence notes:

• None of the three training datasets carries a non-commercial or no-derivatives term on its card (checked 2026-10-10). ODC-By requires attribution; the NOSIBLE dataset is credited above.
• The NOSIBLE labels were produced by language models from several vendors and its text is collected from public news sites. Anyone planning commercial use should review that themselves; the adapter licence below does not settle it.
• The robustness figures on this card were measured on other datasets that are restricted: Financial PhraseBank (CC-BY-NC-SA-3.0), FOMC communication (CC-BY-NC-4.0) and a gold-news set (CC-BY-NC-ND-4.0), plus an oil-headline set (CC-BY-4.0) and a crypto set (MIT per card metadata). Those datasets were used for inference only. The adapter was not trained on them, no text from them is distributed here, and only aggregate rates are reported. Licences as recorded in markets_manifest.json in the repository.

The adapter is released under Apache-2.0, the licence of the base model.

Training procedure

• Command: python train_news.py Qwen/Qwen2.5-0.5B-Instruct news-0.5b
• LoRA: r 16, alpha 32, dropout 0.05, no bias, target modules down_proj, gate_proj, k_proj, o_proj, q_proj, up_proj, v_proj (adapter_config.json).
• Objective: cross-entropy over the three label-token logits at the last prompt position.
• Optimiser: AdamW, peak learning rate 2e-4, weight decay 0.01, one-cycle schedule with 5% warm-up, gradient clipping at 1.0, seed 0 (train_news.py).
• Batch size 16, gradient accumulation 1 (from the command above; not stored in the run summary).
• 719 optimiser steps, one pass over the data, token window 256 (train_news-0.5b.json: steps, max_len).
• Base weights loaded in bfloat16; gradient checkpointing on.
• Training time: 3,988 s (about 66 min) (train_news-0.5b.json, train_seconds).
• Hardware: a laptop GPU, per the repository README. The run summary does not record the hardware.
• Library versions pinned in the repository: peft 0.21.1, transformers 5.18.0, torch 2.14.1.

Evaluation results

On the headline set, bearish recall is 0.083 (recall_bearish): the set holds only 12 negative headlines, so negative-class performance on headlines is not established. A separate run on the same 267 headlines, made by the robustness script, reads 77.2% (robustness_equities.json, clean_accuracy); the two scripts batch and pad differently and the cause of the small difference has not been isolated.

How to read these numbers:

• The NOSIBLE labels are machine-made and the adapter trained on the same kind of labels, so accuracy on NOSIBLE splits overstates agreement with human judgement.
• The headline set is the only one with manually validated labels. It has 267 items, so its margin of error is about 5 points.
• One training run with one seed. There are no confidence intervals on these accuracies.

Robustness: the model can be steered by planted text

A sentence naming a label was appended to each input, and the model's answer was compared with its answer on the unedited text. The metric is the forced rate as defined in the repository README: of the items the model did not already give the planted label, the share pushed to it (JSON field forced_rate_excl_already_target).

Each cell is the lowest and highest value over five markets (equities, oil, gold, central-bank statements, crypto), 300 items per market, sampling seed 0. Sources: robustness_markets.json (instruction row) and robustness_markets_variants.json (other rows), model lora_news_0.5b.

Plainly: this model is vulnerable to planted text. The most effective phrasing here, "Annotation" with X = Bullish, moved 67.7-92.3% of the items that did not already have that label. On the 267 manually labeled headlines the instruction phrasing gives: Bearish 1.1%, Bullish 1.1%, Neutral 92.4% (robustness_equities.json).

The instruction rows were repeated with two more sampling seeds (robustness_markets_seed1.json, robustness_markets_seed2.json). For the Neutral instruction the ranges are seed 1: 18.0-85.0%; seed 2: 22.4-83.5%. The other three phrasings were run with seed 0 only.

These are label-change rates. Nothing here measures trading losses or any real-world harm.

Re-evaluation (2026-10-28)

This adapter (unchanged) was re-scored with the evaluation used for the v2 release of fin-lora-news-qwen2.5-0.5b-adv: test items that also appear in the training data (after lower-casing, stripping punctuation and links) removed, bfloat16, the text truncated instead of the prompt, 95% bootstrap intervals over items. "After planting" appends one of the demo Space's four sentences naming a wrong label (both wrong labels; tweets: a seeded 1,000 items).

A retrained candidate for this repo (all tweets, two epochs, no planted augmentation) raised tweets macro-F1 from 0.834 to 0.874 (paired change +4.0 points, 95% CI [2.5, 5.4]), but its accuracy after planting fell on headlines (46.0% to 39.0%) and news snippets (72.3% to 65.9%), so it was not published. The adversarially trained v2 adapter is better on both counts.

Limitations

• Vulnerable to planted label text, as shown above. Do not feed it untrusted text and act on the output.
• Small test set with human labels (267 headlines, 12 of them negative). Results on negative headlines are not reliable.
• Most training labels are machine-made (NOSIBLE).
• English only. Short text only: prompts are truncated at 256 tokens from the right, which removes the final Sentiment: cue on very long inputs. Trim the text, not the prompt, if your inputs are long.
• One base model size, one training seed, one pass over the data.
• The three classes are a coarse view of sentiment and say nothing about magnitude or time horizon.

How to use

The prompt template and token window ship with the adapter in prompt.json and must be reproduced exactly.

import json
from pathlib import Path

import torch
from huggingface_hub import snapshot_download
from peft import PeftConfig, PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

ADAPTER = "fin-lora-news-qwen2.5-0.5b"  # a local folder, or "9mark9/fin-lora-news-qwen2.5-0.5b" once published
LABELS = ["Bearish", "Bullish", "Neutral"]

adapter_dir = Path(ADAPTER) if Path(ADAPTER).is_dir() else Path(snapshot_download(ADAPTER))
prompt_cfg = json.loads((adapter_dir / "prompt.json").read_text())
base_id = PeftConfig.from_pretrained(str(adapter_dir)).base_model_name_or_path  # Qwen/Qwen2.5-0.5B-Instruct

device = "mps" if torch.backends.mps.is_available() else "cuda" if torch.cuda.is_available() else "cpu"
tok = AutoTokenizer.from_pretrained(base_id)
tok.padding_side = "left"
base = AutoModelForCausalLM.from_pretrained(base_id, dtype=torch.bfloat16)  # older transformers: torch_dtype=
model = PeftModel.from_pretrained(base, str(adapter_dir)).eval().to(device)

label_ids = [tok(" " + label, add_special_tokens=False).input_ids[0] for label in LABELS]


@torch.no_grad()
def classify(texts):
    prompts = [prompt_cfg["template"].format(text=t) for t in texts]
    enc = tok(prompts, return_tensors="pt", padding=True, truncation=True, max_length=prompt_cfg["max_len"]).to(device)
    logits = model(**enc).logits[:, -1, :][:, label_ids]
    probs = torch.softmax(logits.float(), dim=-1).cpu()
    return [(LABELS[int(p.argmax())], [round(float(x), 3) for x in p]) for p in probs]


print(classify(["Shares fell 12% after the company cut its full-year revenue forecast."]))

This mirrors the repository's own loader (finlora/arena.py, finlora/common.py): the tokenizer comes from the base model, padding is on the left, and the class is the argmax over the first tokens of " Bearish", " Bullish" and " Neutral".

Repository

https://github.com/shaikn6/fin-lora