Ornith-1.0-9B — ASHQ1 Quantization
🆕 MTP version now available — wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF with MTP-aware quantization, v6 classifier, and better quality across all sizes.
ASHQ1 quantizations of Ornith-1.0-9B — a state-of-the-art open-source model for agentic coding by DeepReinforce.
Quants
Note: File names contain "BF16" for HuggingFace parser compatibility — these are ASHQ1 quants, not BF16.
| File | Size | Method | Description |
|---|---|---|---|
Ornith-1.0-9B-BF16-ASHQ1-4850.gguf |
4.8 GB | ASHQ1 | Tiny — quality between i1-Q4_K_M and i1-Q5_K_M, 32% smaller than i1-Q5_K_M |
Ornith-1.0-9B-BF16-ASHQ1-5500.gguf |
5.4 GB | ASHQ1 | Compact — beats i1-Q6_K (7.0 GB), 23% smaller |
Ornith-1.0-9B-BF16-ASHQ1-6500.gguf |
6.4 GB | ASHQ1 | Balanced — near i1-Q8_0 (9.6 GB), 33% smaller |
Ornith-1.0-9B-BF16-ASHQ1-7300.gguf |
7.2 GB | ASHQ1 | Best — surpasses i1-Q8_0 (9.6 GB), 25% smaller |
ASHQ1 has been validated across multiple architectures (Qwen3.5-9B, Gemma4-12B QAT, Mellum2 MoE) and consistently outperforms uniform Q6_K and manual SHQ hybrids — see wepiqx/ASHQ1 for method, source code, and full benchmark results. Perplexity comparisons per model are redundant; the priority queue optimises for PPL directly.
Real-World Test: Autonomous Agentic Coding
We tested the 6500 quant (6.4 GB) as the LLM backend for Pi, an autonomous coding agent that runs on llama.cpp. The task: build a complete personal finance dashboard — single HTML file, no external dependencies, with Canvas charts, budget tracker, dark mode, transaction filtering, and upcoming bills.
At temperature 0.6 (not cherry-picked, this was the first run), the agent's workflow was:
- Architecture planning — parsed the requirements, planned 6 sections (overview, charts, transactions, budget, bills, theme toggle)
- Code generation — wrote the full ~1100-line single HTML file with all CSS variables, Canvas rendering, and JavaScript logic
- Self-debugging — caught
date.now is not a function, corrected it todate.getTime(), fixed dark mode toggle logic, adjusted chart rendering to userequestAnimationFrame - Validation — wrote Node.js check scripts to verify: JavaScript syntax, all 30 transactions present, dark mode CSS variables exist, all 6 render functions wired up, script tags balanced
- Delivery — confirmed the
finance-dashboard.htmlpassed all checks and was ready to open in a browser
The model never hallucinated APIs, never broke the single-file constraint, never produced dead code. Every fix it applied was correct. The final output was a polished, production-quality single-page application — built autonomously by a 6.4 GB quant driving an agent.
Full agent interaction log: pi-log-temp0.6-6500.md (589 lines). Final output: finance-dashboard.html (1100 lines, 39.8 KB).
The benchmarks didn't lie. This quant preserves enough quality for autonomous code generation.
Usage
llama.cpp
# Chat mode with built-in Jinja template (recommended)
llama-cli \
-m Ornith-1.0-9B-BF16-ASHQ1-6500.gguf \
--jinja \
-ngl 99 \
-c 8192
# Or with a direct prompt
llama-cli \
-m Ornith-1.0-9B-BF16-ASHQ1-6500.gguf \
-p "Write a Python function that..." \
-ngl 99 \
-c 8192
Recommended sampling: temperature 0.6–1.0, top_p 0.95, top_k 20. The real-world Pi agent test (finance dashboard) used 0.6 with excellent results — lower temperatures are safe.
Ollama
The model's built-in Jinja template is auto-detected by Ollama — no need to specify TEMPLATE manually.
- Create a
Modelfile:
FROM ./Ornith-1.0-9B-BF16-ASHQ1-6500.gguf
PARAMETER num_ctx 8192
PARAMETER temperature 0.6
PARAMETER top_k 20
PARAMETER top_p 0.95
- Build and run:
ollama create ornith-ashq1-6500 -f Modelfile
ollama run ornith-ashq1-6500
LM Studio
- Open LM Studio
- Drag
Ornith-1.0-9B-BF16-ASHQ1-4850.gguf(or 5500/6500/7300) into the app - Set GPU Offload to 99 layers
- Set context length to 8192+
- Set preset sampling:
temperature 0.6,top_p 0.95,top_k 20 - Start chatting
Quantization Configs
Generated by ASHQ1 priority queue — files in this repo were produced with these exact llama-quantize arguments.
4850 Config (base=Q5_K_M)
--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(31)\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.(31)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30))\.ffn_down=Q3_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.(0|25|(?:28|29|30))\.ssm_alpha=Q5_K"
--tensor-type "(blk|BLK)\.(0|25|(?:28|29|30))\.attn_qkv=Q5_K"
--tensor-type "(blk|BLK)\.(0|25|(?:28|29|30))\.ssm_beta=Q5_K"
--tensor-type "(blk|BLK)\.(3|27|31)\.attn_q=Q5_K"
--tensor-type "(blk|BLK)\.(3|27|31)\.attn_v=Q5_K"
--tensor-type "(blk|BLK)\.(0|25|(?:28|29|30))\.attn_gate=Q5_K"
--tensor-type "(blk|BLK)\.(3|27|31)\.attn_k=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|24|26)\.attn_gate=Q4_K"
--tensor-type "(blk|BLK)\.(7|11|15|19|23)\.attn_k=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|24|26)\.ssm_alpha=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|24|26)\.ssm_beta=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|24|26)\.attn_qkv=Q4_K"
--tensor-type "(blk|BLK)\.(7|11|15|19|23)\.attn_q=Q4_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27)\.attn_output=Q4_K"
--tensor-type "(blk|BLK)\.(7|11|15|19|23)\.attn_v=Q4_K"
--tensor-type "(blk|BLK)\.(28|30)\.ssm_out=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|29)\.ssm_out=Q3_K"
--tensor-type ".*output_norm.*=F16"
5500 Config (base=Q5_K_M)
--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(31)\.attn_output=Q6_K"
--tensor-type "(blk|BLK)\.(31)\.ffn_down=Q5_K"
--tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q6_K"
--tensor-type "(blk|BLK)\.(3|7|19|23|27|31)\.attn_v=Q6_K"
--tensor-type "(blk|BLK)\.(3|7|19|23|27|31)\.attn_k=Q6_K"
--tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q6_K"
--tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q6_K"
--tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q6_K"
--tensor-type "(blk|BLK)\.(3|7|19|23|27|31)\.attn_q=Q6_K"
--tensor-type "(blk|BLK)\.(27)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30))\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.ssm_beta=Q5_K"
--tensor-type "(blk|BLK)\.((?:22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q5_K"
--tensor-type "(blk|BLK)\.((?:22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q5_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.ssm_alpha=Q5_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.attn_gate=Q5_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.attn_qkv=Q5_K"
--tensor-type "(blk|BLK)\.(11|15)\.attn_v=Q5_K"
--tensor-type "(blk|BLK)\.(11|15)\.attn_k=Q5_K"
--tensor-type "(blk|BLK)\.(11|15)\.attn_q=Q5_K"
--tensor-type "(blk|BLK)\.((?:28|29|30))\.ssm_out=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21))\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26))\.ssm_out=Q4_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23)\.attn_output=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21))\.ffn_up=Q4_K"
--tensor-type ".*output_norm.*=F16"
6500 Config (base=Q5_K_M)
--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(31)\.ffn_down=Q8_0"
--tensor-type "(blk|BLK)\.(31)\.attn_output=Q8_0"
--tensor-type "(blk|BLK)\.(0|[5-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k=Q8_0"
--tensor-type "(blk|BLK)\.(0|[5-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_v=Q8_0"
--tensor-type "(blk|BLK)\.(0|[5-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q=Q8_0"
--tensor-type "(blk|BLK)\.(0|[5-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q8_0"
--tensor-type "(blk|BLK)\.(27)\.attn_output=Q6_K"
--tensor-type "(blk|BLK)\.((?:29|30))\.ffn_down=Q6_K"
--tensor-type "(blk|BLK)\.(6|(?:10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q6_K"
--tensor-type "(blk|BLK)\.(24|(?:28|29|30))\.ssm_out=Q6_K"
--tensor-type "(blk|BLK)\.(6|(?:10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|4)\.attn_qkv=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|4)\.ssm_beta=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|4)\.ssm_alpha=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|4)\.attn_gate=Q6_K"
--tensor-type "(blk|BLK)\.(22|(?:26|27|28))\.ffn_down=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21)|(?:23|24|25))\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.(15|19|23)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.([4-5]|[7-9])\.ffn_gate=Q5_K"
--tensor-type "(blk|BLK)\.(18|(?:20|21|22)|(?:25|26))\.ssm_out=Q5_K"
--tensor-type "(blk|BLK)\.([4-5]|[7-9])\.ffn_up=Q5_K"
--tensor-type "(blk|BLK)\.(3|7|11)\.attn_output=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17))\.ssm_out=Q4_K"
--tensor-type "(blk|BLK)\.([0-3])\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.([0-3])\.ffn_up=Q4_K"
--tensor-type ".*output_norm.*=F16"
7300 Config (base=Q5_K_M)
--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.((?:29|30|31))\.ffn_down=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q8_0"
--tensor-type "(blk|BLK)\.((?:12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q8_0"
--tensor-type "(blk|BLK)\.((?:12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q8_0"
--tensor-type "(blk|BLK)\.((?:28|29|30))\.ssm_out=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_v=Q8_0"
--tensor-type "(blk|BLK)\.(27|31)\.attn_output=Q8_0"
--tensor-type "(blk|BLK)\.((?:22|23|24|25|26|27|28))\.ffn_down=Q6_K"
--tensor-type "(blk|BLK)\.(18|(?:20|21|22)|(?:24|25|26))\.ssm_out=Q6_K"
--tensor-type "(blk|BLK)\.((?:3|4|5|6|7|8|9|10|11))\.ffn_gate=Q6_K"
--tensor-type "(blk|BLK)\.((?:3|4|5|6|7|8|9|10|11))\.ffn_up=Q6_K"
--tensor-type "(blk|BLK)\.(11|15|19|23)\.attn_output=Q6_K"
--tensor-type "(blk|BLK)\.(7)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.(3)\.attn_output=Q4_K"
--tensor-type "(blk|BLK)\.((?:17|18|19|20|21))\.ffn_down=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16))\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.(0|(?:8|9|10)|(?:12|13|14)|(?:16|17))\.ssm_out=Q5_K"
--tensor-type "(blk|BLK)\.([0-2])\.ffn_up=Q5_K"
--tensor-type "(blk|BLK)\.([0-2])\.ffn_gate=Q5_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6])\.ssm_out=Q4_K"
--tensor-type ".*output_norm.*=F16"
Hardware Requirements
| Variant | Size (GGUF) | VRAM (ctx 8192) | Minimum GPU |
|---|---|---|---|
| 4850 | 4.8 GB | ~5.9 GB | 8 GB |
| 5500 | 5.4 GB | ~6.5 GB | 8 GB |
| 6500 | 6.4 GB | ~7.5 GB | 8 GB |
| 7300 | 7.2 GB | ~8.3 GB | 12 GB |
Notes
- Quantized with the Ornith imatrix by bartowski (248 entries, 6 datasets)
- Architecture: Qwen3.5-9B (32 layers, GQA, SSM, 262k context)
- MIT licensed upstream — no usage restrictions
- Quantization and tuning by wepiqx
- Built with llama.cpp