FinSight Insight 1.5B (GGUF)

GGUF quantizations of a LoRA fine-tune of Qwen2.5-1.5B-Instruct (adapter merged) that rewrites structured personal-finance facts (spending insights, what-if savings simulations, savings-goal milestones) into short, warm, plain-text messages, using only the numbers it is given. Adapter and full details: itsonlyfranz/finsight-insight-qwen2.5-1.5b. Data: itsonlyfranz/finsight-insight-synthetic.

Source code, training and evaluation scripts, and how to reproduce: github.com/itsonlyfranz/finsight-insight-llm.

Which file to pick

Held-out test set, n=246; greedy decoding; Apple M2 Pro 32 GB; llama-server (llama.cpp Metal) over HTTP.

• Q5_K_M is the pick: 1.1 GB, no loss in number fidelity or format validity.
• Q8_0 adds only fact coverage (0.913 vs 0.894) for 46% more bytes.
• Q4_K_M is the smallest and fastest, but it is the only quantization that breaks the format (98.0%) and it has the lowest fact coverage (0.846).

For reference, the unquantized f16 GGUF (not uploaded here) scored fact coverage 0.920, ROUGE-L 0.448, 47.1 tok/s. Number fidelity = pass rate of a port of the app's money-figure guard. Format valid = raw output follows the app's output contract. Fact coverage = share of reference money figures the output states.

How to run

llama.cpp

hf download itsonlyfranz/finsight-insight-qwen2.5-1.5b-GGUF finsight-insight-qwen2.5-1.5b-Q5_K_M.gguf --local-dir .

llama-cli -m finsight-insight-qwen2.5-1.5b-Q5_K_M.gguf --jinja --temp 0 \
  -sys "You are FinSight, a concise personal-finance coach for early-career earners. Use only the numbers provided; never invent amounts, rates, products or guarantees. All amounts are in Philippine pesos (₱); write them with the ₱ sign, never with a dollar sign. Plain text only, no markdown, no emojis." \
  -p "Monthly take-home income: ₱62,500.00
Insight type: risk nudge
Headline: Recurring Subscription Drift
Findings: Monthly recurring subscriptions currently sit at ₱3,737.29. Overlapping media or AI coding copilot tools add up fast.
Suggested action: Audit active tools and downgrade duplicate streaming or SaaS subscriptions to free up ~₱500-₱1,100/mo.
Amount at stake: ₱3,737.29

Rewrite the findings and the suggested action in a warm, specific voice for the user. Reply in exactly two lines:
INSIGHT: <at most two sentences>
ACTION: <one concrete sentence>"

To serve over HTTP use llama-server -m <file> --jinja.

LM Studio

Search for itsonlyfranz/finsight-insight-qwen2.5-1.5b-GGUF in the model search, pick the Q5_K_M file, and download it. In a chat, put the system prompt above in the System Prompt field and send the facts block plus instruction as the user message. Set temperature to 0 to match the reported results.

Prompt format and use

The user message is a facts block followed by one instruction (insight: two lines INSIGHT: / ACTION:; simulation: two to three sentences; goal: one sentence of at most 30 words). Full prompt format and training details are on the adapter page. Always validate the output with a number guard (every money figure must appear in the facts) and fall back to deterministic text on failure.

Out of scope: financial advice, tax or investment guidance, any input that is not in the facts format, other currencies or languages.

Limitations

• Trained and tested only on synthetic data from one generator; real-world robustness is unmeasured.
• References are templated, so reference-based metrics reward matching a house style. No LLM-as-judge was run.
• Number fidelity of 100% on 246 prompts is not a guarantee; keep the guard.
• Single run/seed, no confidence intervals; no Android on-phone measurement was made.
• tok/s and latency were measured on one M2 Pro via llama-server over HTTP.
• Not financial advice.

License

Apache-2.0, inherited from the base model Qwen2.5-1.5B-Instruct.