Ornith-1.5-35B-A3B-Abliterated-GGUF

Official Dynamic GGUF quantization suite of alztrk/Ornith-1.5-35B-A3B-Abliterated (derived from ornith-ai/Ornith-1.5-35B-A3B).

Overview

Ornith-1.5-35B-A3B-Abliterated is created through directional ablation surgery across all 40 hybrid layers, orthogonally projecting refusal directions out from attention and MLP down-projection matrices.

This dedicated repository provides optimized Dynamic GGUF quantizations for local inference in Ollama, LM Studio, and Llama.cpp.

---

Quantization Details

| File Name | Format | Size | Description |

| :--- | :--- | :--- | :--- |

| Ornith-1.5-35B-Abliterated-Dynamic-Q4_K_M.gguf | Q4_K_M (Dynamic) | ~19.71 GB | Balanced performance for 12GB - 16GB VRAM GPUs. |

| Ornith-1.5-35B-Abliterated-Dynamic-Q5_K_M.gguf | Q5_K_M (Dynamic) | ~23.03 GB | Higher precision retention. |

| Ornith-1.5-35B-Abliterated-Dynamic-Q3_K_M.gguf | Q3_K_M (Dynamic) | ~15.61 GB | Lower memory footprint. |

| Ornith-1.5-35B-Abliterated-Q8_0.gguf | Q8_0 | ~34.37 GB | High precision reference quantization. |

---

How to Run

1. Using Ollama

ollama run hf.co/alztrk/Ornith-1.5-35B-A3B-Abliterated-GGUF:Ornith-1.5-35B-Abliterated-Dynamic-Q4_K_M.gguf

2. Using LM Studio / Llama.cpp

llama-cli.exe -hf alztrk/Ornith-1.5-35B-A3B-Abliterated-GGUF -hff Ornith-1.5-35B-Abliterated-Dynamic-Q4_K_M.gguf -p "<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n" -ngl 24 -c 8192

---

License & Attribution

Derived from ornith-ai/Ornith-1.5-35B-A3B, licensed under the Apache 2.0 / MIT License.