GGUF quantized releases of ai9stars/G9v3-3B for llama.cpp and compatible runtimes.
Q4_K_M
Recommended for most users. Excellent balance of quality, memory usage, and speed.
Q5_K_M or Q6_K
For users seeking maximum quality while still benefiting from quantization.
IQ4_NL
Excellent quality-per-GB and one of the strongest modern importance-aware quantizations.
IQ3_M or Q3_K_M
Good balance of usability and reduced memory requirements.
The following importance-aware quantizations are included:
These quantizations were generated using an importance matrix (imatrix) calibration pass and typically provide improved quality retention compared to traditional quantization methods at similar file sizes.
./llama-cli \
-m G9v3-3B-Q4_K_M.gguf \
-p "Hello!"
Create a Modelfile:
FROM ./G9v3-3B-Q4_K_M.gguf
Then:
ollama create g9v3-3b -f Modelfile
ollama run g9v3-3b
Download the desired GGUF file and import it directly into LM Studio.
This repository contains converted GGUF files only.
Please refer to the original model repository for licensing terms, training methodology, benchmark results, intended use, limitations, and safety information.
Original model:
有文件?提交种子。
提交种子ai9stars/G9v3-3B