Dhīr 2B quantised for llama.cpp, for English and Indian languages. This repository holds a 4-bit and an 8-bit build. At just 1.3 GB, the 4-bit build is made for edge deployment: it runs on mobiles, embedded chips and laptops without a GPU, in very little memory. At 2.2 GB, the 8-bit build runs on laptops, mobiles and embedded chips, with output almost identical to the original model.
ZenteiQ · AIKosh · BrahmAI family · Apache 2.0
Dhir-2B-Q4_K_M.gguf (1.3 GB), Dhir-2B-Q8_0.gguf (2.2 GB)| Version | Size | Best for |
|---|---|---|
zenteiq-ai/Dhir-2B-GGUF:Q4_K_M |
1.3 GB | Edge devices, mobiles and embedded chips with little memory |
zenteiq-ai/Dhir-2B-GGUF:Q8_0 |
2.2 GB | Laptops and mobiles, closest to the original |
zenteiq-ai/Dhir-5B-A1.7B-GGUF:Q4_K_M |
3.4 GB | Laptops and desktops; stronger and faster than the 2B on a CPU |
zenteiq-ai/Dhir-5B-A1.7B-GGUF:Q8_0 |
5.4 GB | Laptops, desktops and small GPUs, closest to the original |
Until Dhīr support is merged into llama.cpp, build the dhir-support branch:
git clone -b dhir-support https://github.com/zenteiq-research/llama.cpp
cd llama.cpp
cmake -B build # add -DGGML_CUDA=ON for an NVIDIA GPU
cmake --build build -j --target llama-server llama-cli
Then run the model by its repository name and build; llama.cpp downloads the file on first use:
# Server with an OpenAI-compatible API on port 8080
./build/bin/llama-server -hf zenteiq-ai/Dhir-2B-GGUF:Q4_K_M --jinja --port 8080
# Chat in the terminal
./build/bin/llama-cli -hf zenteiq-ai/Dhir-2B-GGUF:Q4_K_M --jinja
Use :Q8_0 in place of :Q4_K_M for the 8-bit build.
On a CPU, set -t to your number of performance cores and -tb to your total number of cores for the best speed, for example -t 8 -tb 24.
pip install -U langchain-openai
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
base_url="http://localhost:8080/v1",
api_key="EMPTY",
model="zenteiq-ai/Dhir-2B-GGUF",
temperature=0.36,
top_p=0.93,
presence_penalty=0.57,
frequency_penalty=0.02,
extra_body={"top_k": 40, "min_p": 0.0, "repeat_penalty": 1.026},
)
print(llm.invoke("भारत की राजधानी के बारे में दो वाक्य लिखिए।").content)
The model needs about 1.3 GB (4-bit) or 2.2 GB (8-bit) plus 0.11 MB for each token of conversation. For long conversations on devices with little memory, an 8-bit context cache halves the second part:
./build/bin/llama-server -hf zenteiq-ai/Dhir-2B-GGUF:Q4_K_M --jinja --port 8080 -fa on -ctk q8_0 -ctv q8_0
We tested each file against the original model on WikiText-2 perplexity, on the KL divergence of its next-token predictions over 1.5 million tokens in 11 languages, code and mathematics, and on standard benchmarks.
| Build | Perplexity | Same next token as the original |
|---|---|---|
4-bit (Q4_K_M) |
within 3.4% of the original | 91% of the time |
8-bit (Q8_0) |
within 0.1% of the original | 98% of the time |
For the full evaluation, see our technical paper and zenteiq.ai/research/blog.
| File | Size | SHA-256 |
|---|---|---|
Dhir-2B-Q4_K_M.gguf |
1.3 GB | c3c36fe99e06fc768eac5a441359785188d6be621f5beb7b5b12a0b55a57dfeb |
Dhir-2B-Q8_0.gguf |
2.2 GB | 1576cf1d75a34af6c84349dd3fdbd9db1b7af313a19f323ab380b4233725c949 |
Apache 2.0, the same as the original model. The LICENSE and NOTICE files of zenteiq-ai/Dhir-2B are included in this repository.
Released under the Apache 2.0 licence through AIKosh, Hugging Face and ZenteiQ.
ZenteiQ acknowledges the support of the IndiaAI Mission, Ministry of Electronics and Information Technology (MeitY), under the Foundation Model initiative.
@misc{zenteiq2026dhir,
title = {Dh\={\i}r : Sovereign Compact Foundation Models },
author = {{ZenteiQ AiTech Innovations Pvt. Ltd.}},
year = {2026},
howpublished = {Technical report forthcoming}
}
ZenteiQ | India
zenteiq-ai/Dhir-2B