This is a quantized version of MiniCPM5-1B using mradermacher's imatrix.

Commands used (llama.cpp commit 9777256):

pip3.13 install -r requirements.txt
python3.13 convert_hf_to_gguf.py --outfile ./MiniCPM_GGUF.gguf --outtype f32 .;
llama-quantize --imatrix MiniCPM5-1B-SFT.imatrix.gguf MiniCPM_GGUF.gguf MiniCPM5-1B-Q5_K_M.gguf Q5_K_M;

These files were obtained from llama.cpp:

  • requirements.txt
  • convert_hf_to_gguf.py

The initial safetensor was from the MiniCPM5 repository along with other necessary files like the config.json.

To do this conversion yourself, download everything in the repository except the files labeled MiniCPM5-1B-Q5_K_M.gguf and MiniCPM5-1B-F32.gguf.

Then run the commands above after building llama.cpp, changing the paths as necessary.