Transaction Classifier — Fine-tuned MiniLM (v4)

A fine-tuned sentence-transformers/all-MiniLM-L6-v2 model that classifies raw bank transaction strings into 10 budget categories using standard cross-entropy fine-tuning.

This is version 4 (Phase 4b) in a progressive model development series. It was the production model before being succeeded by the metadata-enriched variant (v7).

Model Details

Categories

Performance

Evaluated on 505 unique real-world RBC transactions (3,113 weighted, 2019-2026). Results shown are after Phase 4b preprocessing fixes.

Overall

Per-Category Accuracy

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "maaz-zaidi/transaction-classifier-minilm"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

categories = [
    "Food & Dining", "Transportation", "Shopping & Retail",
    "Entertainment & Recreation", "Healthcare & Medical",
    "Utilities & Services", "Financial Services", "Income",
    "Government & Legal", "Charity & Donations"
]

text = "UBER TRIP HELP.UBER.COM ON"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=64)

with torch.no_grad():
    logits = model(**inputs).logits
    predicted = torch.argmax(logits, dim=-1).item()

print(f"Category: {categories[predicted]}")
# Output: Category: Transportation

Training Data

• Primary: mitulshah/transaction-categorization - 3.6M records, 8K sampled for training (gated dataset)
• Evaluation: 505 real-world RBC bank transactions (2019-2026)

Key Improvements Over Previous Versions

• v3 (SetFit) -> v4: Switched from contrastive learning to standard cross-entropy fine-tuning. Accuracy improved from 80.5% to 84.5%.
• Phase 4b fixes: Preprocessing improvements (AMZN MKTP -> AMAZON MARKETPLACE mapping, ATM/mobile deposit/card fee markers). Accuracy improved from 84.5% to 86.5%.
• Utilities & Services jumped from 34.2% to 68.4%.

Part of a Series

See the Transaction Classifier collection for all 7 model versions.

Limitations

• Trained on only 8,000 samples from a synthetic dataset
• Charity & Donations: 0% accuracy due to insufficient training examples
• Domain-specific to Canadian banking transaction formats
• Best results achieved within a multi-stage pipeline (direction detection + rules + merchant KB + ML)

Citation

@misc{zaidi2026txnclassifier,
  title={Transaction Classifier: Multi-Stage Bank Transaction Categorization},
  author={Maaz Zaidi},
  year={2026},
  url={https://huggingface.co/maaz-zaidi/transaction-classifier-minilm}
}