Turkish-LLM-32B-Instruct-GGUF
GGUF quantizations of Turkish-LLM-32B-Instruct.
Part of the Turkish LLM Family.
Available Quantizations
| File | Size | Use Case |
|---|---|---|
| Q4_K_M | 19GB | Best balance of quality and size. Recommended for most users. |
| Q5_K_M | 22GB | Higher quality, slightly larger. |
| Q8_0 | 33GB | Near-original quality. Requires 40GB+ VRAM. |
| F16 | 62GB | Full precision. Research use. |
Usage with Ollama
# Download and run
ollama run hf.co/ogulcanaydogan/Turkish-LLM-32B-Instruct-GGUF:Q4_K_M
Usage with llama.cpp
./llama-cli -m Turkish-LLM-32B-Instruct-Q4_K_M.gguf -p "Turkiye'nin ekonomik durumu hakkinda bilgi ver." -n 256
Benchmark Results
| Benchmark | Base Model | Turkish-LLM-32B | Delta |
|---|---|---|---|
| MMLU-TR | 0.6518 | 0.6564 | +0.46 |
| XNLI-TR | 0.4578 | 0.4610 | +0.32 |
| XCOPA-TR | 0.6800 | 0.6740 | -0.60 |