Qwen3.5-9B · CuteHuman (GGUF)

The emoji-first 9B persona, quantized for llama.cpp — Q4_K_M at 5.8 GB, vision included.

_{Q4_K_M 5.78 GB  ·  mmproj F16 918 MB  ·  262K context in metadata  ·  text + image  ·  Apache-2.0}

---

🍰 Want it turned up? → Qwen3.8-27B · CuteHuman

The same persona on a 27B model (~2.8× the parameters): richer, and cuter — longer emoji and kaomoji runs, more playful asides, and it can keep a bit going far longer before it runs out of steam.

It ships as a 42 MB LoRA for Qwen/Qwen3.8-27B (21 MB GGUF adapter). This 9B is a single full-model download — start here if VRAM is tight, move up when you want the fuller version.

🖼 Showcase

_{中文 · “在吗” → 边聊边写 C++ hello world，然后开始劝你吃苦瓜
English · small talk that remembers the pastry shop, then volunteers to help with your project}

---

📦 Files

---

🚀 Quickstart

The GGUF carries no chat template (43 metadata keys, no tokenizer.chat_template), so pass the one in this repo — otherwise the server falls back to a generic prompt and the persona sounds off.

llama-server \
  -m Qwen3.5-9B-CuteHuman-SFT-Q4_K_M.gguf \
  --mmproj mmproj-Qwen3.5-9B-CuteHuman-SFT-F16.gguf \
  --chat-template-file chat_template.jinja \
  -c 32768 -ngl 99 -fa on --jinja \
  --host 0.0.0.0 --port 18080

Then talk to it like any OpenAI endpoint:

curl -s http://127.0.0.1:18080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
        "messages": [{"role": "user", "content": "在吗"}],
        "temperature": 0.7, "top_p": 0.8, "max_tokens": 256,
        "chat_template_kwargs": {"enable_thinking": false}
      }'

llama-cli, one shot:

llama-cli -m Qwen3.5-9B-CuteHuman-SFT-Q4_K_M.gguf --chat-template-file chat_template.jinja \
  -p "在吗" -n 256 --temp 0.7 --top-p 0.8

Images: add --mmproj as above and pass an image in the message content; drop the --mmproj line if you only need text and want to save 918 MB of RAM.

---

💡 Notes

• Thinking is on by default in the template. For chit-chat send "chat_template_kwargs": {"enable_thinking": false}, otherwise the reasoning trace eats the token budget before the cute part starts.
• The metadata says 262,144 context, but the KV cache is what actually limits you — 32K on a 12–16 GB card is a realistic start.
• The persona lives in the weights, not in a system prompt: an empty system prompt reproduces the screenshots.
• Emoji and kaomoji are the default register; a system prompt (“少用表情” / “tone the emoji down”) dials it back.
• Converted from the merged bf16 checkpoint of wesjos/Qwen3.5-9B-CuteHuman — go there for the full-precision weights and the transformers recipe.
• qwen35 includes a 1-layer MTP head (nextn_predict_layers = 1); leave speculative decoding off unless you have measured a win on your own hardware.

⚠️ Limitations

• Q4_K_M is a 4-bit quant: expect a small quality drop against the bf16 original, most visible on long reasoning.
• 9B knowledge ceiling — facts, math and long-form reasoning are 9B level; the 27B sibling is where that gap closes.
• It will happily keep a bit going; cut it off with a system prompt when you need a straight answer.
• A persona tune, not a safety tune: review outputs before putting it in front of users.

---

📁 Related

_{Demo screenshots live in demos/.}