Banglish Companion — GGUF

Quantized builds of banglish-companion. What it is and why it exists are on that page. This one is about running it.

Two builds below. You need one of them — 1.06 GB for the smaller. No account, no network calls once it is on disk. You will need Python and llama-cpp-python.

There is a newer model. banglish-companion-3b is built on a 3B base and answers shorter and closer to the question. On a Mac, banglish-companion-3b-mlx runs it at 3.6 GB. These files stay published and working; start there if you are choosing today.

Banglish e

kon file ta nibo? duitar moddhe je kono ekta. Q4_K_M chhoto (1.06 GB), Q8_0 boro (1.82 GB) kintu beshi precise. tomar computer er upor depend kore.

ki lagbe? Python ar pip install llama-cpp-python. account lagbe na, taka lagbe na, chalate internet lagbe na. tumi ja bolba, tomar computer er baire jabe na.

mone rakho: ek-duita message e bhalo, lomba adda te na.

bipod hole? kichu message er uttor model dey na — companion.py age oigulo dekhe ney, tomar kichu korte hobe na.

Which File

On a Mac? There is an 8-bit MLX build at bangla-banglish-companion-mlx. It runs natively on Apple silicon. If you want the smallest file, though, stay here — 4-bit holds up better in GGUF than it does in MLX.

Both are published so you can pick. Q4_K_M is the smaller download and needs less memory; Q8_0 keeps more of the original precision. We compared them by hand on the same prompts and did not find a difference we would ask you to care about — so take whichever suits your machine.

Use it

pip install llama-cpp-python huggingface_hub
hf download sifat-febo/bangla-banglish-companion-gguf --local-dir banglish
python banglish/companion.py

That is the whole thing: it answers, then keeps talking. In your own code, from companion import reply is the only import you need. It picks up whichever .gguf file sits beside it.

What answers you, and when

Some messages are answered from nirapotta.py rather than by the model, and reply() checks that first on every turn. It is plain Python, needs nothing installed, and runs in microseconds.

Verify Your Download

A truncated GGUF opens without complaint and then generates noise. It does not error. Check the file you got:

shasum -a 256 banglish-companion-v2-*.gguf

5c9ba1cddd3e2fdadf0435de3c423016ec2dbae94d0dc705f1cd9bc294a24a43  Q4_K_M
7288d0256b7aa771381db592ce822bf9b865f61be21a2b5cb075f4cd40ab757c  Q8_0

What Quantization Cost

We checked both files against the weights they came from before publishing. Neither lost anything we could see.

Limits

Neither limit comes from quantization. Both are in the parent model and both survive into these files.

It degrades after about four turns. Give it conversation history and it repeats itself and loses the thread. Use it for single exchanges.

It deflects rather than refuses. On a harmful request it tends to change the subject instead of declining outright. That is not the same thing as a refusal. Do not put this anywhere a refusal matters.

Some answers are not the model's. A few kinds of message are answered from a file in this repo, before the weights are reached.

AI Disclosure

The author designed the system, chose the base models, ran all training and measurement on the author's own machine, read the models' conversations, and chose what to publish and what to withhold. Claude Code (Anthropic) was used as a coding and writing tool under that direction.

License

Apache 2.0

Cite the report, not the packaging — these files carry the parent's weights unchanged, and a quantization format is not a separate contribution. The report is at https://doi.org/10.5281/zenodo.22371896.

@misc{febo2026banglishcompanion,
  author    = {Febo, Sifat},
  title     = {Banglish Companion: Small Chat Models for Romanized Bengali
               — Technical Report},
  year      = {2026},
  month     = sep,
  publisher = {Zenodo},
  version   = {1.0},
  doi       = {10.5281/zenodo.22371896},
  url       = {https://doi.org/10.5281/zenodo.22371896},
  note      = {Repository: sifat-febo/bangla-banglish-companion-gguf}
}