Piccolo-9B-BF16
Piccolo-9B-BF16 is the BF16 GGUF release of Piccolo-9B for local tool review and context selection.
Download
hf download agentscope-ai/Piccolo-9B-BF16 --local-dir Piccolo-9B-BF16
Precision and conversion
The text backbone was converted directly from the full checkpoint to BF16 GGUF. Download all 4 shards together and load Piccolo-9B-BF16-00001-of-00004.gguf.
Required decision inference
Use the Piccolo-compatible llama.cpp runtime with bidirectional prompt attention and native candidate scoring. The /v1/systemone endpoint returns probabilities without generating answer tokens.
Numerical validation
On four test questions, the largest candidate-probability difference from the full checkpoint was 0.001873. Validation details document the comparison.
Evaluation
Piccolo-9B benchmark results · All models
This model is also available on ModelScope.