Ornith-1.5-9B GGUF
GGUF quantizations of https://huggingface.co/ornith-ai/Ornith-1.5-9B for use with llama.cpp, LM Studio, Ollama (GGUF import), KoboldCpp, and other GGUF-compatible runtimes.
About the Base Model
Ornith-1.5-9B is a 9B-parameter Qwen3.5-family model using a hybrid architecture consisting of:
- 32 transformer layers
- Hybrid linear attention (Gated DeltaNet) and full attention
- MRoPE positional encoding
- Dense architecture (no routed experts)
These GGUF files are text-only conversions intended for inference in llama.cpp-compatible software.
Conversion Notes
- Converted using upstream
llama.cppwith native Qwen3.5 support. - The model was converted directly to Q8_0 GGUF and all other quants were generated from that Q8_0 source.
- Importance matrix (imatrix) calibration was generated using WikiText-2 and applied to IQ quantizations.
- The original Hugging Face configuration advertises an MTP/next-token prediction layer that is not present in the released weights. GGUF metadata was patched so the layer count matches the actual tensors contained in the model.
Available Quantizations
Standard Quants
- Q2_K
- Q3_K_S
- Q3_K_M
- Q3_K_L
- Q4_0
- Q4_1
- Q4_K_S
- Q4_K_M
- Q5_K_S
- Q5_K_M
- Q6_K
- Q8_0
IQ Quants
- IQ2_M
- IQ3_XXS
- IQ3_XS
- IQ3_M
- IQ4_XS
- IQ4_NL
Quant Recommendations
| Quant | Recommended Use |
|---|---|
| IQ2_M / Q2_K | Maximum memory savings |
| Q3_K_M | Very low RAM systems |
| Q4_K_M | Best balance of quality and size |
| Q5_K_M | High-quality daily use |
| Q6_K | Near-original quality |
| Q8_0 | Highest quality quant available in this repository |
Usage with llama.cpp
./llama-cli \
-m Ornith-1.5-9B-Q4_K_M.gguf \
-p "Explain quantum computing in simple terms."