English | 中文

MiniCPM5-2B-Math-GGUF

GGUF builds of MiniCPM5-2B-Math, a 2.5B math and proof specialist, for llama.cpp, Ollama and LM Studio. Pair it with the MiniCPM-Math Harness to run a small proof swarm on a laptop.

Converted with llama.cpp b10964 (convert_hf_to_gguf.py, tokenizer pre-type minicpm5) and quantized with llama-quantize. The recommended sampling settings (temperature 0.9, top-p 0.95, min-p 0, top-k off) are embedded in the GGUF metadata, so llama.cpp uses them by default. All results on the model card were measured with the BF16 safetensors on vLLM.

llama.cpp

llama-server -hf Caldalis/MiniCPM5-2B-Math-GGUF:Q8_0 --jinja -ngl 99 -c 98304 -np 4 -kvu --port 8000

• --jinja uses the embedded chat template (thinking mode, <think> … </think>).
• -np 4 -kvu gives 4 parallel slots that share a 96K-token KV cache: one slot per agent for the harness. Run the harness with --profile quick against this server.
• --port 8000 is where the harness looks by default; without it, llama-server listens on 8080.
• The embedded defaults already set min_p=0; llama.cpp's usual 0.05 can encourage repetition with this model.

Ollama

ollama run hf.co/Caldalis/MiniCPM5-2B-Math-GGUF:Q8_0

Start the server with a larger context window (for example OLLAMA_CONTEXT_LENGTH=65536 ollama serve), because proofs routinely need tens of thousands of tokens. With the harness, pass --base-url http://127.0.0.1:11434/v1 --model hf.co/Caldalis/MiniCPM5-2B-Math-GGUF:Q8_0 --no-raw.

Recommended settings

temperature=0.9, top_p=0.95, min_p=0.0, thinking enabled, and a large max_tokens. The published benchmark numbers let every call use the full 131,072-token context: a quarter of HMMT samples think past 64K tokens, and many of those still end with the right answer. On a laptop, 16K–32K is a practical compromise.

License

Apache-2.0 (see LICENSE).