Qwen3.8-27B · Mario-HTML LoRA (GGUF)

The HTML-generating LoRA as a 21 MB GGUF adapter — mount it on any Qwen3.8-27B quant at runtime, no merge.

_{21 MB f16 adapter  ·  128 tensors  ·  α 32 → runtime scale 2.0  ·  Qwen3.8-27B base  ·  Apache-2.0}

> Note: thinking effort xhigh is not good for this lora

---

🖼 Showcase

you may need some prompt to trigger model's html generation like:You are Mario-html. You are a helpful assistant who generates nice visual content.

Your browser does not support the video tag — download the 28 s clip.
_{28 s screen recording · .ai-view pages streaming in — cards, callouts, a comparison table, inline SVG; you can see the next prompt being queued at the bottom. Status line at the time: 2448 tokens at ~42 tok/s.
“what does sigmoid look like” → formula block, an SVG plot with asymptotes and the inflection point, three property cards, a reference table
“can you show me what a sloth looks like.” → thinking panel, an SVG sloth hanging from a branch, two fact cards, a “Did you know?” callout}

---

📦 Files

---

🚀 Quickstart

Prerequisite: any Qwen3.8-27B GGUF as the base (Qwen3.8-27B-UD-Q4_K_M.gguf recommended; add mmproj-F16.gguf if you want vision).

llama-server \
  -m Qwen3.8-27B-UD-Q4_K_M.gguf \
  --lora mario-html-27b-lora-f16.gguf \
  --mmproj mmproj-F16.gguf \
  --chat-template-file chat_template.jinja \
  -c 131072 -ngl 99 -fa on --jinja \
  --alias qwen3.8-27b --host 0.0.0.0 --port 18080

⚠️ --chat-template-file chat_template.jinja is mandatory. The LoRA converter does not write a chat template into the adapter GGUF (this file has 10 metadata keys, no tokenizer.chat_template); llama.cpp falls back to the base GGUF's template, which is the stock one and has no trigger sentence. Without it the model never enters HTML mode.

Verify the adapter is mounted:

curl -s http://127.0.0.1:18080/lora-adapters
# [{"id":0,"path":".../mario-html-27b-lora-f16.gguf","scale":1.0,...}]

Verify the template took effect (should print 1):

curl -s http://127.0.0.1:18080/props | grep -c "You are Mario-html"

Toggle the adapter at runtime — no restart, no reload:

curl -X POST http://127.0.0.1:18080/lora-adapters -H 'Content-Type: application/json' -d '[{"id":0,"scale":0.0}]'  # off
curl -X POST http://127.0.0.1:18080/lora-adapters -H 'Content-Type: application/json' -d '[{"id":0,"scale":1.0}]'  # on

WebUI

cd webui
pip install -r requirements.txt
WEBUI_PORT=8798 API_URL=http://127.0.0.1:18080 MODEL_ALIAS=qwen3.8-27b ./run_webui.sh
# → http://<host>:8798

Reasoning switch (low / medium / high) next to the input, stored in localStorage; left pane chat, right pane live .html-canvas render (it recognises .ai-view / .card / .modern-table); “🖼 Add Image” works when the backend runs with --mmproj; the system prompt can be overridden at the bottom (empty = the trigger sentence from the template).

The frontend speaks plain OpenAI /v1/chat/completions; the only extra field is:

{ "chat_template_kwargs": { "enable_thinking": true, "reasoning_effort": "low" } }

reasoning_effort can also sit at the top level of the request (llama.cpp maps it into the template). Omitted = medium (the template's new default, down from xhigh).

---

🎛 Sampling

{
  "temperature": 1.0, "top_p": 0.95, "top_k": 20, "min_p": 0.0,
  "max_tokens": 16384,
  "chat_template_kwargs": { "enable_thinking": true, "reasoning_effort": "medium" }
}

• The output is an HTML fragment (.ai-view / .card / .modern-table), not a full <!DOCTYPE html> page. Ask explicitly for a complete document, or edit the system sentence, if that's what you need.
• Don't enable MTP speculative decoding (--spec-type draft-mtp): the draft head was trained on the base model, and hit rate collapses once this LoRA is on — measurably slower.

---

📊 Measured (RTX 4090 24 GB, Q4_K_M + this LoRA)

---

🧪 Training

Same adapter as the PEFT repo, converted to GGUF:

• Base Qwen/Qwen3.8-27B (4-bit nf4 during training), QLoRA r16 / α32, q/k/v/o only, 10.5M trainable (0.04%)
• 830 HTML instruction samples (730 with CoT), seq 5120, lr 2e-4, 2 epochs, bs 1 × grad-accum 16
• No system prompt at train time; CoT was context but masked out of the loss → it changes how it writes, not how it thinks
• One RTX 4090, 2.09 h / 103 optimizer steps

---

⚠️ Limitations

• Form, not facts: content accuracy is still whatever the base model knows.
• Attention layers only — heavy nesting and interactive JS are beyond it.
• Context ≥ 32768; xhigh thinking can eat 8k–10k tokens per turn.
• Training data is mostly English; Chinese prompts work with slightly looser layout stability.

---

📁 Related