ScarletKc-2.0-GGUF
ScarletKc/ScarletKc-2.0, an open-source 27B multimodal reasoning and chat model created by independent developer KC (scarletkc.com), quantized to GGUF for llama.cpp and compatible runtimes.
| File | Size | Precision | Role |
|---|---|---|---|
ScarletKc-2.0-Q4_K_M.gguf |
16.7 GB (15.6 GiB) |
4-bit (Q4_K_M, 4.98 BPW) |
27B hybrid language & reasoning model (Qwen3_5TextModel) |
ScarletKc-2.0-mmproj-F16.gguf |
928 MB (0.86 GiB) |
16-bit (F16) |
Multimodal vision projector (Qwen3VLVisionModel); optional for text-only use |
Architecture details, alignment method, and limitations are documented on the main model card.
Usage
Requires a recent llama.cpp build with qwen35 architecture support.
OpenAI-compatible server (text-only)
llama-server \
-m ScarletKc-2.0-Q4_K_M.gguf \
-ngl 99 \
-c 16384 \
--jinja \
--temp 0.6 --top-p 0.9 --top-k 20 \
--port 8080
OpenAI-compatible server (vision + text)
llama-server \
-m ScarletKc-2.0-Q4_K_M.gguf \
--mmproj ScarletKc-2.0-mmproj-F16.gguf \
-ngl 99 \
-c 16384 \
--jinja \
--temp 0.6 --top-p 0.9 --top-k 20 \
--port 8080
CLI single-turn check
llama-cli \
-m ScarletKc-2.0-Q4_K_M.gguf \
-p "你是谁?介绍一下你自己吧" \
--temp 0.6 --top-p 0.9 --top-k 20 \
--single-turn
- Sampling: Avoid greedy decoding (
--temp 0) on open-ended conversation; usetemperature 0.6–0.7,top_p 0.9,top_k 20. - Thinking mode: Enabled by default via the embedded chat template (
<think>...</think>). To request direct replies without reasoning traces throughllama-server, pass"chat_template_kwargs": {"enable_thinking": false}in the request body.
Conversion
Exported from the bf16 weights with llama.cpp (b1-5676dfb):
- Language model converted via
convert_hf_to_gguf.py --outtype bf16 --no-mtpand quantized withllama-quantize ... Q4_K_M(themtp.*speculative decoding head is omitted in GGUF for runtime compatibility and retained in the BF16 safetensors). - Vision projector exported via
convert_hf_to_gguf.py --mmproj --outtype f16.
License
Apache License 2.0, like ScarletKc/ScarletKc-2.0, JonathanColetti/Qwen3.8-27B-Uncensored, and Qwen/Qwen3.8-27B.