A QLoRA fine-tune of Qwen/Qwen3.8-27B (Apache-2.0,
Qwen3.5 MoE, qwen35 arch in llama.cpp) on a devops / coding / security corpus, quantized to GGUF.
One variant additionally has refusal directions ablated ("abliterated").
| File | Quant | Size | Lineage |
|---|---|---|---|
Qwen3.8-27B-DevSecOps-QLoRA-Q4_K_M.gguf |
Q4_K_M | 16 GB | merged fine-tune, no ablation |
Qwen3.8-27B-DevSecOps-QLoRA-abliterated-Q4_K_M.gguf |
Q4_K_M | 16 GB | merged fine-tune, refusal-ablated |
optim-Q3_K_M.gguf |
Q3_K_M | 13 GB | refusal-ablated (the daily-driver quant) |
optim-Q2_K.gguf |
Q2_K | 10 GB | refusal-ablated, smallest |
Qwen3.8-27B-DevSecOps-QLoRA-mmproj-F16.gguf |
F16 | 885 MB | CLIP vision projector for the multimodal tower |
All quants carry the embedded Qwen chat template; llama-server --jinja works out of the box.
# 3-bit, refusal-ablated (the config this was served with)
llama-server -hf outbackdingo/Qwen3.8-27B-DevSecOps-QLoRA:optim-Q3_K_M.gguf \
--host 0.0.0.0 --port 8000 -ngl 99 -c 128000 --jinja -fa on
# with vision
llama-server -hf outbackdingo/Qwen3.8-27B-DevSecOps-QLoRA:Qwen3.8-27B-DevSecOps-QLoRA-abliterated-Q4_K_M.gguf \
--mmproj outbackdingo/Qwen3.8-27B-DevSecOps-QLoRA:Qwen3.8-27B-DevSecOps-QLoRA-mmproj-F16.gguf -ngl 99 -c 32768 --jinja
convert_hf_to_gguf → llama-quantize| Metric | base Qwen3.8-27B | this fine-tune |
|---|---|---|
| Refusal rate | 31.7% | 15.0% |
| Mean token F1 vs reference | 0.2200 | 0.2185 |
| ROUGE-L | 0.1216 | 0.1187 |
| Mean answer length | 306 words | 354 words |
| F1 higher than base | — | 30 / 60 prompts |
Honest read: the fine-tune roughly halves refusals and answers longer, but n-gram overlap with the reference answers is flat (differences are within noise at n=60). It is not measurably "smarter" by these metrics — it is more permissive and more verbose in the security domain. No MMLU/GSM8K-style benchmarks were run.
Apache-2.0, inherited from the base model. Base weights: Qwen/Qwen3.8-27B.
Qwen/Qwen3.8-27B