Turkish-LLM-14B-Instruct-GGUF
GGUF quantized versions of Turkish-LLM-14B-Instruct, a 14.7B parameter Turkish language model fine-tuned from Qwen2.5-14B-Instruct.
Available Quantizations
| File | Quant | Size | RAM Needed | Best For |
|---|---|---|---|---|
| Turkish-LLM-14B-Instruct-F16.gguf | F16 | 28 GB | 32-35 GB | Full precision, A100/H100, 2x 24GB GPU |
| Turkish-LLM-14B-Instruct-Q8_0.gguf | Q8_0 | 15 GB | 18-20 GB | 32GB RAM, RTX 3090/4090 |
| Turkish-LLM-14B-Instruct-Q5_K_M.gguf | Q5_K_M | 9.8 GB | 13-14 GB | 16GB+ RAM, M2/M3 Mac |
| Turkish-LLM-14B-Instruct-Q4_K_M.gguf | Q4_K_M | 8.4 GB | 11-12 GB | 16GB RAM laptop, M1/M2 Mac |
Recommended Quantization
- Q4_K_M for most consumer hardware (best size/quality ratio)
- Q5_K_M if you have extra RAM and want slightly better quality
- Q8_0 for minimal quality loss with enough RAM
- F16 for research/evaluation with high-end hardware
Usage
llama.cpp
# Download a quantization
huggingface-cli download ogulcanaydogan/Turkish-LLM-14B-Instruct-GGUF Turkish-LLM-14B-Instruct-Q4_K_M.gguf
# Run inference
./llama-cli -m Turkish-LLM-14B-Instruct-Q4_K_M.gguf \
-p "<|im_start|>system\nSen yardimci bir Turkce yapay zeka asistanisin.<|im_end|>\n<|im_start|>user\nTurkiye'nin baskenti neresidir?<|im_end|>\n<|im_start|>assistant\n" \
-n 256 --temp 0.7
Ollama
ollama run hf.co/ogulcanaydogan/Turkish-LLM-14B-Instruct-GGUF:Q4_K_M
LM Studio
Download any GGUF file and load it directly in LM Studio.
Chat Template
This model uses the ChatML format:
<|im_start|>system
Sen yardimci bir Turkce yapay zeka asistanisin.<|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant
{assistant_response}<|im_end|>
Model Details
- Base Model: Turkish-LLM-14B-Instruct (Qwen2.5-14B-Instruct + SFT)
- Parameters: 14.7B
- Architecture: Qwen2, 48 layers, 5120 hidden size
- Context Length: 32,768 tokens
- Vocabulary: 152,064 tokens
- Quantized with: llama.cpp
Related
- Turkish-LLM-14B-Instruct - Original model (safetensors)
- Turkish-LLM-7B-Instruct - Lighter 7B variant
- Turkish-LLM-14B-Chat - Live demo