UDGAM LABS · GGUF · Q8_0 163 MB

Laghu-GGUF

GGUF builds of Laghu, a 152M-parameter English chat model trained from scratch by Udgam Labs for $146 of pretraining compute. It thinks before it answers (on/off), holds multi-turn conversations and chats. These files run it in Ollama, LM Studio, llama.cpp and other GGUF runtimes. Use Q8_0 (163 MB): it matches the full-precision model within measurement noise. On a Mac mini (Apple M6) it generates about 330 tokens per second in llama.cpp and needs about 154 MiB of memory at a 4K context.

Try it: in your browser, on your device · code on GitHub · udgamlabs.com/models/laghu

Quick start

Ollama (one line; Ollama picks up Laghu's chat format and sampling settings from this repository):

ollama run hf.co/udgamlabs/Laghu-GGUF:Q8_0 --think=false

In a chat, /set nothink turns thinking off and /set think turns it on; in the API, send "think": false.

LM Studio: search for udgamlabs/Laghu-GGUF in LM Studio and download Q8_0, or click "Use this model" → LM Studio on this page. LM Studio shows the thinking separately from the answer. Through its API, send "reasoning_effort": "none" to turn thinking off.

llama.cpp (use the Jinja chat template; without --jinja the server rejects requests):

llama-server -hf udgamlabs/Laghu-GGUF:Q8_0 --jinja --reasoning off -c 16384
# OpenAI-compatible API on http://localhost:8080
curl -s localhost:8080/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Give me two tips for staying focused."}],"temperature":0.7,"top_p":0.95,"top_k":64}'

Remove --reasoning off to let Laghu think before answering (it writes its reasoning first; this helps a little on short arithmetic and hurts most other tasks).

Ollama with the Modelfile (offline, or to name the model yourself): download Laghu-Q8_0.gguf and Modelfile into one folder, then

ollama create laghu -f Modelfile
ollama run laghu --think=false "Who are you?"

Sampling defaults: temperature 0.7, top-p 0.95, top-k 64. Keep context at 16,384 tokens or less.

Results

Quantisation quality (measured by us on 2026-10-22: 32 windows of 2,048 tokens of held-out text, compared with the F16 file; task checks through llama-server, greedy, thinking off):

We do not publish 4- and 5-bit files: they lose clearly, and because Laghu's width (640) is not a multiple of 256, llama.cpp's K-quants fall back to simpler types and barely shrink the file. The task scores differ from the model card's because this check uses a shorter output limit and only the first items; it compares formats, not models.

Faithfulness: the F16 file matches the transformers model almost exactly (mean KL 0.0000018, top-1 agreement 99.89%), and llama.cpp applies Laghu's 512-token sliding window (the local layers' cache stays at 10.5 MiB at any context length). The chat format was checked in llama.cpp and Ollama (thinking off and on, multi-turn, tool calls) and in LM Studio 0.4.26 (thinking off and on, multi-turn).

For the model's own results (benchmarks, chat tests against SmolLM2-135M-Instruct and Falcon-H1-Tiny-90M-Instruct), see the Laghu model card.

Where it falls short

• These are files of a small research model: Laghu is weak at maths (GSM8K 7.9% with thinking) and instruction following (IFEval 26.1), knows little, and can loop with greedy decoding. Details on the model card.
• llama.cpp, Ollama, LM Studio and the MLX server turn thinking on by default; use the switches above for normal chat.
• llama-server caps the context at 16,384 tokens (32K prompts are rejected).

How it was trained

See the Laghu model card and the technical report. Converted with llama.cpp's convert_hf_to_gguf.py (commit d7a695e) from the released safetensors; the Q8_0 file is byte-identical when rebuilt.

Intended use and limitations

Local, offline use and research. Not for factual answers or decisions that matter; light safety training.

Licence

Apache-2.0. Copyright 2026 Udgam Labs LLP. The chat template inside the files is Google's Gemma 4 template (Apache-2.0).

Citation

@misc{udgamlabs2026laghu,
  title  = {Laghu: a fully open 152M chat model trained from scratch},
  author = {Prasad, Jitesh K},
  year   = {2026},
  howpublished = {Udgam Labs},
  url    = {https://huggingface.co/udgamlabs/Laghu}
}

---

About Udgam Labs. Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. udgamlabs.com