Piccolo-4B-Q8_0
Piccolo-4B-Q8_0 is the Q8_0 GGUF release of Piccolo-4B for local tool review and context selection.
Download
hf download agentscope-ai/Piccolo-4B-Q8_0 --local-dir Piccolo-4B-Q8_0
Precision and conversion
The text backbone was converted to BF16 GGUF, then quantized to Q8_0. The full checkpoint retains the vision and auxiliary tensors.
Required decision inference
Use the Piccolo-compatible llama.cpp runtime with bidirectional prompt attention and native candidate scoring. The /v1/systemone endpoint returns probabilities without generating answer tokens.
Numerical validation
On four test questions, the largest candidate-probability difference from the full checkpoint was 0.00564096. Validation details document the comparison.
Evaluation
Piccolo-4B benchmark results · All models
This model is also available on ModelScope.