Sakura · GGUF quantization of Qwen/Qwen2.5-0.5B-Instruct

Independent community quantization. Not an official release of the upstream author.

1. Model Overview

This repository is produced by the local HF Quant Agent control plane. It records only checks that actually ran. Unmeasured quality, speed and compatibility are marked untested.

2. Original Model / Attribution

• Source: Qwen/Qwen2.5-0.5B-Instruct
• Publisher of this quantization: webmp3, when a release is actually uploaded
• Upstream authors retain their license and attribution
• GGUF source publisher: Qwen/Qwen2.5-0.5B-Instruct-GGUF
• GGUF source commit: 9217f5db79a29953eb74d5343926648285ec7e67
• GGUF source filename: qwen2.5-0.5b-instruct-q8_0.gguf
• GGUF source SHA-256: ca59ca7f13d0e15a8cfa77bd17e65d24f6844b554a7b6c12e07a5f89ff76844e
• GGUF source provenance verified: True

3. Source Revision

Pinned revision: 7ae557604adf67be50417f59c2c2f167def9a775

Jobs are bound to this SHA. A later upstream commit does not silently change a running or completed job.

4. Supported GGUF Quantizations

Requested: Q3_K_M

5. Quantization Methodology

Route: native_gguf_stream

Adapter: standard-llamacpp

This route performs existing-GGUF requantization from the pinned Q8_0/F16/BF16 source GGUF identified above. This job did not convert the upstream BF16/safetensors checkpoint. Source tensor storage measured in the GGUF: F32: 121, Q8_0: 170.

No importance-matrix calibration was run by this job. Requantizing a previously quantized source can add quantization error. Estimated sizes are planning ratios, not measurements, unless a file byte count is listed above.

The target quant label describes the selected stock quantizer preset. Stock llama.cpp may retain floating-point tensors or use fallback tensor types; every tensor is not necessarily stored in the named target type. The table records measured output tensor types and only reports a fallback count when the quantizer printed one.

Source tensor count: 291. Source full tensor-name set SHA-256: 70e194dd80c4c853c75f2073b7209f1ca5c93265d2ed7bd1836f82ac67bcc8b0. New and reused outputs must retain the source tensor count and complete name set.

6. Architecture Information

• model_type / architecture: qwen2
• parameters: 494032768
• experts: n/a or not declared
• source formats observed: safetensors, bfloat16
• custom code present: False
• trust_remote_code executed: False

7. Conversion and Runtime Compatibility

• GGUF structure valid: True
• Runtime load tested: True
• Inference tested: True

8. Download and Usage Instructions

Use a llama.cpp build that supports this architecture. Example, after you confirm compatibility:

llama-cli -m <file>.gguf -p "Hello" -n 32

This card does not claim that every runtime can load the file.

9. Validation Results

All selected streamed output compositions passed structural checks and one stock CPU inference per quant. Every composed shard was hashed before and after inference.

10. Limitations

• No coding benchmark, perplexity or quality score is claimed unless a measured value is copied from this job's own log.
• Shard resume and upload verification apply only to artifacts listed in this card.

11. License Information

Source license: apache-2.0

Decision: apache-2.0 allows redistribution with attribution

If the license does not allow redistribution, this card must not be uploaded.

12. Reproduction Information

• Control plane: HF Quant Agent
• Planner route: native_gguf_stream
• Workflow: none
• Blockers: none recorded