Yejin Korean 3B v2 - SFT (GGUF)
GGUF quantized versions of cooler8/yejin-korean-3b-v2-sft for use with llama.cpp, Ollama, LM Studio, and other GGUF-compatible inference engines.
Model Details
- Base model: cooler8/yejin-korean-3b-v2-sft
- Parameters: ~3B
- Vocabulary: 64,000 tokens (custom Korean tokenizer)
- Architecture: 3072d / 28L / 24H / 8KV
Available Quantizations
| Filename | Quantization | Size | Description |
|---|---|---|---|
| yejin-korean-3b-v2-sft-q4-k-m.gguf | Q4_K_M | 1.73 GB | Medium quality, good balance of size and accuracy |
| yejin-korean-3b-v2-sft-q8-0.gguf | Q8_0 | 2.99 GB | High quality, larger file size |
Recommended Quantization
- Q4_K_M: Best balance of quality and size for most use cases.
- Q8_0: Higher quality, larger file size. Use when accuracy is critical.
Chat format
<s><|user|>
{question}<|end|>
<|assistant|>
{answer}<|end|>
</s>
The GGUF embeds this as tokenizer.chat_template, so llama-server / llama-cli chat mode use it automatically.
Usage with llama.cpp
./llama-cli -m yejin-korean-3b-v2-sft-q4-k-m.gguf -p "한국의 수도는 어디인가요?" -n 256
Usage with Ollama
Create a Modelfile (also shipped in this repo as Modelfile):
FROM ./yejin-korean-3b-v2-sft-q4-k-m.gguf
TEMPLATE """{{ if .System }}<|system|>
{{ .System }}<|end|>
{{ end }}{{ if .Prompt }}<|user|>
{{ .Prompt }}<|end|>
{{ end }}<|assistant|>
{{ .Response }}<|end|>
"""
PARAMETER stop "<|end|>"
PARAMETER stop "</s>"
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1
Then:
ollama create yejin-korean-3b -f Modelfile
ollama run yejin-korean-3b
License
Apache 2.0