Qwen3.5 2B Mermaid LoRA GGUF
Lucasgitguy-co/Qwen3.5-2B-Mermaid-MLX-LoRA converted from MLX to a llama.cpp LoRA file. Load it on top of a Qwen3.5-2B base GGUF with --lora.
| File | Type | Size (GB) |
|---|---|---|
Qwen3.5-2B-Mermaid-LoRA-F16.gguf |
LoRA, rank 8, alpha 16, 186 language-model layers | 0.017 |
Use
Tested base: unsloth/Qwen3.5-2B-GGUF Qwen3.5-2B-Q4_K_M.gguf. Any Qwen3.5-2B GGUF made by a current llama.cpp should work. For images, add mmproj-F16.gguf from the same unsloth repo (llama-mtmd-cli ... --mmproj mmproj-F16.gguf --image diagram.png). The adapter doesn't change the vision tower.
hf download unsloth/Qwen3.5-2B-GGUF Qwen3.5-2B-Q4_K_M.gguf --local-dir .
hf download Lucasgitguy-co/Qwen3.5-2B-Mermaid-LoRA-GGUF Qwen3.5-2B-Mermaid-LoRA-F16.gguf --local-dir .
llama-cli -m Qwen3.5-2B-Q4_K_M.gguf --lora Qwen3.5-2B-Mermaid-LoRA-F16.gguf --jinja \
-p "Write Mermaid code for a flowchart: Start, Process, End."
The adapter was trained in MLX on the 4-bit base mlx-community/Qwen3.5-2B-4bit. On other bases (F16, other quants) its output can differ from what it gave during training.
How it was made and tested
mlx_lora_to_peft.pyturned the MLX weights into a PEFT adapter:lora_aandlora_bwere transposed intolora_Aandlora_B, alpha = scale 2.0 × rank 8 = 16, and the names were mapped to transformers'model.language_model.layers.*. llama.cppconvert_lora_to_gguf.py(checkout1b6935b) then converted that againstQwen/Qwen3.5-2Brevision15852e8.- Tested 2026-11-16 with llama.cpp 1.0.0 (build 12359):
- Base F16 + this LoRA gives exactly the same greedy text as a hand-merged model (W + 2·B·A, converted separately) on 3 Mermaid prompts of 80 tokens each. Both differ from the plain base. This includes the
linear_attnlayers (in_proj_qkv,in_proj_z,in_proj_a,in_proj_b,out_proj). - The LoRA also loads without errors on unsloth's Q4_K_M and gives plain Mermaid answers there.
- Image test: unsloth Q4_K_M +
mmproj-F16.gguf+ this LoRA turned a Start → Process → End flowchart image into the matching Mermaidgraph TD.
- Base F16 + this LoRA gives exactly the same greedy text as a hand-merged model (W + 2·B·A, converted separately) on 3 Mermaid prompts of 80 tokens each. Both differ from the plain base. This includes the
Licence
Apache-2.0, the same as the adapter and Qwen3.5-2B.