CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5-GGUF
This repository contains GGUF quantized versions of:
micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5
These files are optimized for:
- llama.cpp
- Ollama
- LM Studio
- Jan
- google colabs
Available Quantizations
| File | Size | Description |
|---|---|---|
| Q4_K_M | ~986 MB | Best balance between quality and size |
| Q5_K_M | ~1.13 GB | Better quality while remaining compact |
| Q6_K | ~1.27 GB | High quality |
| Q8_0 | ~1.65 GB | Highest quality quantization |
llama.cpp
./llama-cli \
-m codemate-qwen-Q4_K_M.gguf \
-c 32768 \
-p "Explain Python decorators with an example."
Ollama
Create a Modelfile:
FROM ./codemate-qwen-Q8_0.gguf
PARAMETER num_ctx 32768
PARAMETER stop "<|im_end|>"
TEMPLATE \"\"\"
<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Input }}<|im_end|>
<|im_start|>assistant
\"\"\"
Build:
ollama create codemate -f Modelfile
ollama run codemate
Notes
This repository contains quantized versions only.
For the original merged model:
https://huggingface.co/micymike/CodeMate-Qwen-1.5B-32K-Distilled-on-Claude-Fable-5
Disclaimer
This project is an independent research effort and is not affiliated with or endorsed by Anthropic, Claude, Alibaba, or Qwen.
Author
Built by micymike 🇰🇪