fiduciary-qwen3-4b-GGUF
GGUF quants of Fiduciary — a local-first personal-finance advisor LLM (Qwen3-4B-Instruct-2507 fine-tune), for llama.cpp, Ollama, LM Studio, Jan, and everything else in the GGUF universe. Your portfolio questions never leave your machine.
⚠️ Educational tool, not financial advice. Fiduciary explains concepts, reads the portfolio you show it, and talks through trade-offs like a patient advisor — it does not know your full situation and it can be wrong. Decisions are yours.
How do I run a local financial-advisor LLM?
Ollama (fastest path):
ollama run hf.co/albertobarnabo/fiduciary-qwen3-4b-GGUF:Q4_K_M
LM Studio: search fiduciary in the model browser, pick Q4_K_M.
llama.cpp:
llama-cli -hf albertobarnabo/fiduciary-qwen3-4b-GGUF:Q4_K_M \
-p "I'm 28, saving 800€/month. Explain index funds vs picking stocks, for my situation."
Which file should I download?
| file | size | use it when |
|---|---|---|
fiduciary-qwen3-4b-Q4_K_M.gguf |
2.5 GB | default — best size/quality balance, runs on 8 GB RAM |
fiduciary-qwen3-4b-Q5_K_M.gguf |
2.9 GB | a bit more quality headroom, 8–16 GB RAM |
fiduciary-qwen3-4b-Q8_0.gguf |
4.3 GB | near-lossless, 16 GB+ RAM |
Need F16 or another quant for re-quantizing? Open a discussion — happy to add it.
What is this model?
A LoRA fine-tune (rank 16, 16 layers, prompt-masked so only advisor turns train) of Qwen3-4B-Instruct-2507, trained with mlx-lm on synthetic advisor dialogues: portfolio review conversations, tool-calling traces (live prices/news), and financial-literacy explanations. Final validation loss 0.046 — a training-sanity signal on a small validation set, not a quality benchmark; no standardized finance benchmark is claimed here.
These GGUFs were made the standard QLoRA way: adapters (trained against the 4-bit MLX base) fused onto the full-precision Qwen3-4B-Instruct-2507, then quantized with llama.cpp b10092. The chat template is embedded.
The full agent runtime — local portfolio.json, price/news tools, the loop that
makes it an advisor rather than a chatbot — lives in the
GitHub repo. The weights are the
brain; the repo is the owl's perch. 🦉
The Fiduciary family
| artifact | for |
|---|---|
| fiduciary-qwen3-4b | Apple silicon / MLX (4-bit, 2.3 GB) |
| this repo | everyone else: Ollama, LM Studio, llama.cpp, Jan |
| fiduciary-qwen3-4b-lora | the raw LoRA adapter (56 MB) — stack it on your own Qwen3-4B |
Limitations
- English-first; finance terminology skews US/EU retail investing.
- A 4B model: confidently wrong sometimes — verify numbers before acting.
- Knowledge is frozen at base-model training; live data comes from the agent tools, not the weights.
- Trained on synthetic dialogues (generated + curated), not real client conversations.