Sign up

wepiqx/Ornith-1.0-9B-ASHQ1-GGUF

🤗 On Hugging Facetext-generationapache-2.025 GBGGUF✓ HF checksums availableupdated today
Magnet

Ornith-1.0-9B — ASHQ1 Quantization

🆕 MTP version now available — wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF with MTP-aware quantization, v6 classifier, and better quality across all sizes.

ASHQ1 quantizations of Ornith-1.0-9B — a state-of-the-art open-source model for agentic coding by DeepReinforce.

Quants

Note: File names contain "BF16" for HuggingFace parser compatibility — these are ASHQ1 quants, not BF16.

| File | Size | Method | Description |

|------|:----:|:------:|-------------|

| Ornith-1.0-9B-BF16-ASHQ1-4850.gguf | 4.8 GB | ASHQ1 | Tiny — quality between i1-Q4_K_M and i1-Q5_K_M, 32% smaller than i1-Q5_K_M |

| Ornith-1.0-9B-BF16-ASHQ1-5500.gguf | 5.4 GB | ASHQ1 | Compact — beats i1-Q6_K (7.0 GB), 23% smaller |

| Ornith-1.0-9B-BF16-ASHQ1-6500.gguf | 6.4 GB | ASHQ1 | Balanced — near i1-Q8_0 (9.6 GB), 33% smaller |

| Ornith-1.0-9B-BF16-ASHQ1-7300.gguf | 7.2 GB | ASHQ1 | Best — surpasses i1-Q8_0 (9.6 GB), 25% smaller |

ASHQ1 has been validated across multiple architectures (Qwen3.5-9B, Gemma4-12B QAT, Mellum2 MoE) and consistently outperforms uniform Q6_K and manual SHQ hybrids — see wepiqx/ASHQ1 for method, source code, and full benchmark results. Perplexity comparisons per model are redundant; the priority queue optimises for PPL directly.

Real-World Test: Autonomous Agentic Coding

We tested the 6500 quant (6.4 GB) as the LLM backend for Pi, an autonomous coding agent that runs on llama.cpp. The task: build a complete personal finance dashboard — single HTML file, no external dependencies, with Canvas charts, budget tracker, dark mode, transaction filtering, and upcoming bills.

At temperature 0.6 (not cherry-picked, this was the first run), the agent's workflow was:

1. Architecture planning — parsed the requirements, planned 6 sections (overview, charts, transactions, budget, bills, theme toggle)

2. Code generation — wrote the full ~1100-line single HTML file with all CSS variables, Canvas rendering, and JavaScript logic

3. Self-debugging — caught date.now is not a function, corrected it to date.getTime(), fixed dark mode toggle logic, adjusted chart rendering to use requestAnimationFrame

4. Validation — wrote Node.js check scripts to verify: JavaScript syntax, all 30 transactions present, dark mode CSS variables exist, all 6 render functions wired up, script tags balanced

5. Delivery — confirmed the finance-dashboard.html passed all checks and was ready to open in a browser

The model never hallucinated APIs, never broke the single-file constraint, never produced dead code. Every fix it applied was correct. The final output was a polished, production-quality single-page application — built autonomously by a 6.4 GB quant driving an agent.

Full agent interaction log: pi-log-temp0.6-6500.md (589 lines). Final output: finance-dashboard.html (1100 lines, 39.8 KB).

The benchmarks didn't lie. This quant preserves enough quality for autonomous code generation.

Usage

llama.cpp

# Chat mode with built-in Jinja template (recommended)
llama-cli \
  -m Ornith-1.0-9B-BF16-ASHQ1-6500.gguf \
  --jinja \
  -ngl 99 \
  -c 8192

# Or with a direct prompt
llama-cli \
  -m Ornith-1.0-9B-BF16-ASHQ1-6500.gguf \
  -p "Write a Python function that..." \
  -ngl 99 \
  -c 8192

Recommended sampling: temperature 0.6–1.0, top_p 0.95, top_k 20. The real-world Pi agent test (finance dashboard) used 0.6 with excellent results — lower temperatures are safe.

Ollama

The model's built-in Jinja template is auto-detected by Ollama — no need to specify TEMPLATE manually.

1. Create a Modelfile:

FROM ./Ornith-1.0-9B-BF16-ASHQ1-6500.gguf

PARAMETER num_ctx 8192
PARAMETER temperature 0.6
PARAMETER top_k 20
PARAMETER top_p 0.95

2. Build and run:

ollama create ornith-ashq1-6500 -f Modelfile
ollama run ornith-ashq1-6500

LM Studio

1. Open LM Studio

2. Drag Ornith-1.0-9B-BF16-ASHQ1-4850.gguf (or 5500/6500/7300) into the app

3. Set GPU Offload to 99 layers

4. Set context length to 8192+

5. Set preset sampling: temperature 0.6, top_p 0.95, top_k 20

6. Start chatting

Quantization Configs

Generated by ASHQ1 priority queue — files in this repo were produced with these exact llama-quantize arguments.

4850 Config (base=Q5_K_M)

--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(31)\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.(31)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30))\.ffn_down=Q3_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.(0|25|(?:28|29|30))\.ssm_alpha=Q5_K"
--tensor-type "(blk|BLK)\.(0|25|(?:28|29|30))\.attn_qkv=Q5_K"
--tensor-type "(blk|BLK)\.(0|25|(?:28|29|30))\.ssm_beta=Q5_K"
--tensor-type "(blk|BLK)\.(3|27|31)\.attn_q=Q5_K"
--tensor-type "(blk|BLK)\.(3|27|31)\.attn_v=Q5_K"
--tensor-type "(blk|BLK)\.(0|25|(?:28|29|30))\.attn_gate=Q5_K"
--tensor-type "(blk|BLK)\.(3|27|31)\.attn_k=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|24|26)\.attn_gate=Q4_K"
--tensor-type "(blk|BLK)\.(7|11|15|19|23)\.attn_k=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|24|26)\.ssm_alpha=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|24|26)\.ssm_beta=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|24|26)\.attn_qkv=Q4_K"
--tensor-type "(blk|BLK)\.(7|11|15|19|23)\.attn_q=Q4_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27)\.attn_output=Q4_K"
--tensor-type "(blk|BLK)\.(7|11|15|19|23)\.attn_v=Q4_K"
--tensor-type "(blk|BLK)\.(28|30)\.ssm_out=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|29)\.ssm_out=Q3_K"
--tensor-type ".*output_norm.*=F16"

5500 Config (base=Q5_K_M)

--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(31)\.attn_output=Q6_K"
--tensor-type "(blk|BLK)\.(31)\.ffn_down=Q5_K"
--tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q6_K"
--tensor-type "(blk|BLK)\.(3|7|19|23|27|31)\.attn_v=Q6_K"
--tensor-type "(blk|BLK)\.(3|7|19|23|27|31)\.attn_k=Q6_K"
--tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q6_K"
--tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q6_K"
--tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q6_K"
--tensor-type "(blk|BLK)\.(3|7|19|23|27|31)\.attn_q=Q6_K"
--tensor-type "(blk|BLK)\.(27)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30))\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.ssm_beta=Q5_K"
--tensor-type "(blk|BLK)\.((?:22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q5_K"
--tensor-type "(blk|BLK)\.((?:22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q5_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.ssm_alpha=Q5_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.attn_gate=Q5_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.attn_qkv=Q5_K"
--tensor-type "(blk|BLK)\.(11|15)\.attn_v=Q5_K"
--tensor-type "(blk|BLK)\.(11|15)\.attn_k=Q5_K"
--tensor-type "(blk|BLK)\.(11|15)\.attn_q=Q5_K"
--tensor-type "(blk|BLK)\.((?:28|29|30))\.ssm_out=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21))\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26))\.ssm_out=Q4_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23)\.attn_output=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21))\.ffn_up=Q4_K"
--tensor-type ".*output_norm.*=F16"

6500 Config (base=Q5_K_M)

--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(31)\.ffn_down=Q8_0"
--tensor-type "(blk|BLK)\.(31)\.attn_output=Q8_0"
--tensor-type "(blk|BLK)\.(0|[5-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k=Q8_0"
--tensor-type "(blk|BLK)\.(0|[5-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_v=Q8_0"
--tensor-type "(blk|BLK)\.(0|[5-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q=Q8_0"
--tensor-type "(blk|BLK)\.(0|[5-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q8_0"
--tensor-type "(blk|BLK)\.(27)\.attn_output=Q6_K"
--tensor-type "(blk|BLK)\.((?:29|30))\.ffn_down=Q6_K"
--tensor-type "(blk|BLK)\.(6|(?:10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q6_K"
--tensor-type "(blk|BLK)\.(24|(?:28|29|30))\.ssm_out=Q6_K"
--tensor-type "(blk|BLK)\.(6|(?:10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|4)\.attn_qkv=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|4)\.ssm_beta=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|4)\.ssm_alpha=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|4)\.attn_gate=Q6_K"
--tensor-type "(blk|BLK)\.(22|(?:26|27|28))\.ffn_down=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21)|(?:23|24|25))\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.(15|19|23)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.([4-5]|[7-9])\.ffn_gate=Q5_K"
--tensor-type "(blk|BLK)\.(18|(?:20|21|22)|(?:25|26))\.ssm_out=Q5_K"
--tensor-type "(blk|BLK)\.([4-5]|[7-9])\.ffn_up=Q5_K"
--tensor-type "(blk|BLK)\.(3|7|11)\.attn_output=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17))\.ssm_out=Q4_K"
--tensor-type "(blk|BLK)\.([0-3])\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.([0-3])\.ffn_up=Q4_K"
--tensor-type ".*output_norm.*=F16"

7300 Config (base=Q5_K_M)

--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.((?:29|30|31))\.ffn_down=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q8_0"
--tensor-type "(blk|BLK)\.((?:12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q8_0"
--tensor-type "(blk|BLK)\.((?:12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q8_0"
--tensor-type "(blk|BLK)\.((?:28|29|30))\.ssm_out=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_v=Q8_0"
--tensor-type "(blk|BLK)\.(27|31)\.attn_output=Q8_0"
--tensor-type "(blk|BLK)\.((?:22|23|24|25|26|27|28))\.ffn_down=Q6_K"
--tensor-type "(blk|BLK)\.(18|(?:20|21|22)|(?:24|25|26))\.ssm_out=Q6_K"
--tensor-type "(blk|BLK)\.((?:3|4|5|6|7|8|9|10|11))\.ffn_gate=Q6_K"
--tensor-type "(blk|BLK)\.((?:3|4|5|6|7|8|9|10|11))\.ffn_up=Q6_K"
--tensor-type "(blk|BLK)\.(11|15|19|23)\.attn_output=Q6_K"
--tensor-type "(blk|BLK)\.(7)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.(3)\.attn_output=Q4_K"
--tensor-type "(blk|BLK)\.((?:17|18|19|20|21))\.ffn_down=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16))\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.(0|(?:8|9|10)|(?:12|13|14)|(?:16|17))\.ssm_out=Q5_K"
--tensor-type "(blk|BLK)\.([0-2])\.ffn_up=Q5_K"
--tensor-type "(blk|BLK)\.([0-2])\.ffn_gate=Q5_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6])\.ssm_out=Q4_K"
--tensor-type ".*output_norm.*=F16"

Hardware Requirements

| Variant | Size (GGUF) | VRAM (ctx 8192) | Minimum GPU |

|---------|:-----------:|:----------------:|:-----------:|

| 4850 | 4.8 GB | ~5.9 GB | 8 GB |

| 5500 | 5.4 GB | ~6.5 GB | 8 GB |

| 6500 | 6.4 GB | ~7.5 GB | 8 GB |

| 7300 | 7.2 GB | ~8.3 GB | 12 GB |

Notes

  • Quantized with the Ornith imatrix by bartowski (248 entries, 6 datasets)
  • Architecture: Qwen3.5-9B (32 layers, GQA, SSM, 262k context)
  • MIT licensed upstream — no usage restrictions
  • Quantization and tuning by wepiqx
  • Built with llama.cpp