中文Sign up

WhiskyAKM/MiniCPM5-2B-NVFP4-GGUF

🤗 Hugging Face sourcetext-generationapache-2.02B activated1.7 GBGGUF✓ 2 checksumsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo WhiskyAKM/MiniCPM5-2B-NVFP4-GGUF ./model-folder
Needs a seeder →

MiniCPM5-2B NVFP4 GGUF

GGUF quantized version of openbmb/MiniCPM5-2B, the second model in the MiniCPM5 series. It is a dense 2B Transformer (LlamaForCausalLM) built for on-device, local deployment, and resource-constrained scenarios, reaching 2B-class open-source SOTA.

Model Overview

MiniCPM5-2B is designed for local assistants, coding agents, tool-use workflows, and reasoning scenarios where a compact model is preferred. It keeps a small deployment footprint while providing native long-context support (131,072 tokens), and was post-trained with RL + OPD (On-Policy Distillation). It supports a chat template with thinking/reasoning mode (controlled via enable_thinking) and XML-style tool calling.

Model Architecture

Property Value
Architecture LlamaForCausalLM (dense)
Total Parameters 2.52B
Non-Embedding Parameters 1.98B
Layers 42
Attention Heads (GQA) 16 Q / 2 KV
Context Length 131,072 (128K)
Original Precision bfloat16
Supported Languages en, zh

Available GGUF Files

File Quantization Size Use Case
minicpm5-2b-nvfp4.gguf NVFP4 1.4 GB 4-bit NVFP4, compact, quality/size balance
minicpm5-2b-dspark-nvfp4.gguf NVFP4 1.4 GB Draft model

Usage

llama.cpp CLI

./llama-cli \
  -m minicpm5-2b-nvfp4.gguf \
  -p "Explain quantum computing in simple terms." \
  --temp 1.0 --top-p 0.95

llama-server (OpenAI-compatible API)

./llama-server \
  -m minicpm5-2b-nvfp4.gguf \
  --host 0.0.0.0 --port 8080

The GGUF also works with Ollama and LM Studio.

Thinking Mode

The model supports deep-thinking output. You can control it per request via the chat template, e.g. with an OpenAI-compatible API:

"chat_template_kwargs": {"enable_thinking": false}

Tool Calling

MiniCPM5-2B emits XML-style tool calls. Tool definitions are injected into the prompt, and tool results are returned in the observation/tool role. SGLang's built-in minicpm5 parser converts these to OpenAI-compatible tool_calls natively (see upstream model card).

Generation Parameters

Recommended parameters from the original model:

Parameter Value
Temperature 1.0
Top-P 0.95

Quantization

This GGUF was quantized to NVFP4 (a 4-bit floating-point format) from the BF16 source model, verified with llama.cpp's GGUF reader. All weight tensors use NVFP4; the embedding/output and norm tensors are kept in higher precision (Q6_K / F32) to preserve quality. Recommended sampling settings are stored in the file's metadata: temperature 1.0, top-p 0.95.

Acknowledgements

License

Apache-2.0 License