Qwen2.5-0.5B-Instruct-abliterated-v3 — GGUF
GGUF quantizations of huihui-ai/Qwen2.5-0.5B-Instruct-abliterated-v3, an abliterated/uncensored Qwen2.5-0.5B fine-tune, produced with llama.cpp.
Available files
| File | Quantization | Description |
|---|---|---|
model-F16.gguf |
F16 | Full precision (1.0 GB) |
model-Q8_0.gguf |
Q8_0 | 8-bit |
model-Q6_K.gguf |
Q6_K | 6-bit |
model-Q5_K_M.gguf |
Q5_K_M | 5-bit medium |
model-Q5_K_S.gguf |
Q5_K_S | 5-bit small |
model-Q5_0.gguf |
Q5_0 | 5-bit |
model-Q4_K_M.gguf |
Q4_K_M | 4-bit medium (recommended) |
model-Q4_K_S.gguf |
Q4_K_S | 4-bit small |
model-Q4_0.gguf |
Q4_0 | 4-bit |
model-Q3_K_L.gguf |
Q3_K_L | 3-bit large |
model-Q3_K_M.gguf |
Q3_K_M | 3-bit medium |
model-Q3_K_S.gguf |
Q3_K_S | 3-bit small |
model-Q2_K.gguf |
Q2_K | 2-bit |
Usage
With llama.cpp:
./llama-cli -m model-Q4_K_M.gguf -p "Your prompt" -n 256
Or any GGUF-compatible runtime (Ollama, LM Studio, koboldcpp, etc.).
License
Apache License 2.0. Built on Qwen/Qwen2.5-0.5B-Instruct.
Disclaimer
Abliterated/uncensored model. Evaluate before use.