MiniCPM5-2B GGUF

GGUF quantizations of openbmb/MiniCPM5-2B.

Model Description

MiniCPM5-2B is a dense ~2.5B parameter Transformer from the MiniCPM5 series, built for on-device and resource-constrained deployment. LlamaForCausalLM architecture, 131K context, strong at coding, math, tool use, and agentic tasks in its class.

Architecture: LlamaForCausalLM | 42 layers | 2048 hidden | 16 attn heads | 2 KV heads | 130,560 vocab

Quantization

Converted from the official F16 GGUF master using llama.cpp b10842 (CPU-only, no imatrix).

Q4_K (Mixed): Q4_K base with Q6_K on attention and FFN output tensors (llama.cpp's built-in Q4_K_M large-precision treatment).

Usage

Ollama

ollama run hf.co/peasantsmith/MiniCPM5-2B-GGUF:Q4_K

llama.cpp

llama-cli -m MiniCPM5-2B-Q4_K.gguf -p "Your prompt here" -ngl <layers that fit your VRAM>

Python (llama-cpp-python)

from llama_cpp import Llama
llm = Llama(model_path="MiniCPM5-2B-Q4_K.gguf", n_ctx=8192, n_gpu_layers=-1)
output = llm("Your prompt here", max_tokens=256)
print(output["choices"][0]["text"])

Composition

Original Model

• Model: openbmb/MiniCPM5-2B
• GGUF source: openbmb/MiniCPM5-2B-GGUF
• License: Apache 2.0