Piccolo-9B-Q8_0
Piccolo-9B-Q8_0 is the Q8_0 GGUF release of Piccolo-9B for local tool review and context selection.
Download
hf download agentscope-ai/Piccolo-9B-Q8_0 --local-dir Piccolo-9B-Q8_0
Precision and conversion
The text backbone was converted to BF16 GGUF, then quantized to Q8_0. Download all 2 shards together and load Piccolo-9B-Q8_0-00001-of-00002.gguf.
Required decision inference
Use the Piccolo-compatible llama.cpp runtime with bidirectional prompt attention and native candidate scoring. The /v1/systemone endpoint returns probabilities without generating answer tokens.
Numerical validation
On four test questions, the largest candidate-probability difference from the full checkpoint was 0.00286038. Validation details document the comparison.
Evaluation
Piccolo-9B benchmark results · All models
This model is also available on ModelScope.