hishab-qwen3.5-0.8b-pipe-signed-full-v13-trim38k

bf16 weights of Qwen/Qwen3.5-0.8B fine-tuned (full fine-tune, all weights, 3 epochs, lr 3e-05) to turn a Banglish / Bangla / English message about money into one description|amount|category line per transaction:

tuition fee|-5000|tuition
rickshaw|-80|transport
salary|+35000|salary

The amount's sign is the type: - expense, + income. NONE means no transaction. Split each line on its last two |. About half the output tokens of the JSON models.

Trimmed vocabulary. The 248k-token vocabulary of the base model is cut to 38,000 tokens (English, Bangla and Banglish, every single-character token so any text still encodes, and all special tokens); the tied embedding / output table is sliced to match. No retraining: every Hishab test message tokenizes exactly as before. 0.75B to 0.54B parameters, and about 45% faster generation on a Pixel 8. Text in other scripts still works but splits into more, rarer pieces. See vocab_trim in the eval json and ml/hishab/vocab_trim.py in the training repo.

The held-out numbers below were measured on the untrimmed model; the trimmed llama.cpp builds score the same (see the -GGUF repo).

Categories: food, grocery, rent, utilities, mobile_recharge, transport, tuition, salary, gift, loan, shopping, medical, entertainment, other. Type is expense or income.

System prompt used in training (use it verbatim):

You are Hishab, a Bangladeshi personal-finance assistant. The user writes in English, Bangla or Banglish (Bangla in Latin letters), often mixed. Extract every financial transaction they mention, one per line as description|amount|category. amount is a BDT number (expand shorthand like 1.5k or 2 lakh) written with - for an expense and + for income. category is one of: food, grocery, rent, utilities, mobile_recharge, transport, tuition, salary, gift, loan, shopping, medical, entertainment, other. If there is no transaction, reply NONE. Reply with the lines only.

Qwen3.5 is trained and served with thinking disabled (empty <think> block in the assistant turn).

Held-out results (bf16, greedy; none of these rows were seen in training)

Splits of ryuma007/hishab-synthetic-v7. exact = the set of (amount, category, type) tuples equals the gold set; F1 is micro F1 over those tuples.

test_clean and numbers_clean are the rows where two independent labellers (Gemini 3.8 Flash and Claude Sonnet 5.5, following the labelling guideline) agree, with their label as the gold; they are the headline numbers. The original test labels break the guideline on about 14% of rows (plans and stated totals recorded as entries, tutoring income as tuition), so this model scores lower there by design.

Use

from huggingface_hub import hf_hub_download
from tokenizers import Tokenizer
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("ryuma007/hishab-qwen3.5-0.8b-pipe-signed-full-v13-trim38k")
tok._tokenizer = Tokenizer.from_file(hf_hub_download("ryuma007/hishab-qwen3.5-0.8b-pipe-signed-full-v13-trim38k", "tokenizer.json"))  # see below
model = AutoModelForCausalLM.from_pretrained("ryuma007/hishab-qwen3.5-0.8b-pipe-signed-full-v13-trim38k", torch_dtype="bfloat16")
msgs = [{"role": "system", "content": SYSTEM_PROMPT},  # the prompt above, verbatim
        {"role": "user", "content": "Salary paisi 35k but basha vara dilam 12k"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, enable_thinking=False, return_tensors="pt")
out = model.generate(ids, max_new_tokens=384, do_sample=False)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

The tok._tokenizer line matters for Bangla script: transformers 5.10 rebuilds a Qwen2Tokenizer that splits Bengali words at their combining marks, while this model was trained (and llama.cpp runs) with the tokenization in tokenizer.json. English and Banglish are unaffected.

Limitations

• Trained and evaluated on synthetic messages only; real users type messier Banglish.
• Category conventions follow the Hishab labelling guideline (https://github.com/RaihanulHaque/hishab/blob/main/docs/LABELLING_GUIDELINE.md); other apps may want e.g. tutoring income as tuition rather than salary.
• Amounts in spelled-out Banglish are the weakest area (see the numbers rows above).

Quantized GGUF builds for llama.cpp live in ryuma007/hishab-qwen3.5-0.8b-pipe-signed-full-v13-trim38k-GGUF. Training pipeline: https://github.com/RaihanulHaque/hishab (ml/). Full results: eval_hf.json in this repo.