MiMo-Ornith-9B-AGSI-Abliterated-HQ-GGUF

Official GGUF Quantizations | Native 32-Block Header (Zero Missing Tensor Crashes) | Multi-Tier Reasoning & Tool-Use Enabled

---

📌 Model Overview

This repository provides official GGUF quantizations of OliviaRossi/MiMo-Ornith-9B-AGSI-Abliterated-HQ, a 9.7B parameter hybrid reasoning and agentic model synthesized from:

• Xiaomi MiMo-V2.6-Distill-Qwen-9B: Chain-of-thought distillation and mathematical/algorithmic depth.
• DeepReinforce Ornith-1.5-9B: Autonomous terminal execution, bash mastery, and self-correcting agent loops.
• Adaptive Geodesic Spectral Interpolation (AGSI): Manifold-aligned parameter fusion conserving activation variance ($|W|_2$) across all 32 layers.
• Bayesian Refusal Abliteration (Abliterix): Non-destructive refusal vector null-projection constrained to $\text{KL} \le 0.007$.

🛡️ Why These GGUFs Work Out-of-the-Box

Standard GGUF conversions of Qwen 3.5 / Ornith fine-tunes fail in llama.cpp with:

error loading model: check_tensor_dims: tensor 'blk.32.attn_norm.weight' not found

This is caused by upstream configs inheriting "mtp_num_hidden_layers": 1, declaring a phantom 33rd block with no weights behind it.

These GGUF quantizations have been compiled with a neutralized MTP layer count (mtp_num_hidden_layers = 0):

• ✅ Declares the true 32 transformer blocks (qwen35.block_count = 32).
• ✅ Loads directly into llama-server, LM Studio, Ollama, and Unsloth without requiring --override-kv or --no-mtp workarounds.

---

📊 Quantization Breakdown & Hardware Recommendations

---

🧠 Reasoning Spectrum & Thinking Controls

These GGUF files contain the full harmonized chat_template.jinja. When serving via llama-server, you can steer reasoning depth and preserve thinking across turns:

1. Reasoning Effort Steering (reasoning_effort)

Clients can pass "reasoning_effort" via the OpenAI-compatible /v1/chat/completions endpoint:

• "reasoning_effort": "none": Emits <think>\n\n</think>\n\n immediately, bypassing chain-of-thought for fast, direct responses.
• "reasoning_effort": "low": Injects concise reasoning constraints for fast code completions and brief explanations.
• "reasoning_effort": "medium": The default baseline reasoning depth without prompt bloat.
• "reasoning_effort": "high": Injects deep verification instructions for complex debugging and architectural tasks.
• "reasoning_effort": "max": Directs exhaustive exploration of edge cases and alternative hypotheses.

2. Prefix KV Cache Parity (--reasoning-preserve)

Run llama-server with --reasoning-preserve (or pass chat_template_kwargs: {"preserve_thinking": true}). This keeps historical reasoning traces in context across multi-turn chats, maintaining a 100% prefix KV cache hit rate.

---

🚀 Running with llama.cpp

1. Local Server (llama-server)

llama-server \
  -m mimo-ornith-9b-agsi-abliterated-q4_k_m.gguf \
  -c 65536 \
  -ngl 99 \
  --jinja \
  --reasoning on \
  --reasoning-format deepseek \
  --reasoning-preserve \
  --host 0.0.0.0 \
  --port 8080

2. Interactive CLI (llama-cli)

llama-cli \
  -m mimo-ornith-9b-agsi-abliterated-q5_k_m.gguf \
  -c 32768 \
  -ngl 99 \
  --jinja \
  -p "Write a C program to enumerate active network interfaces and their broadcast addresses."

---

🦙 Running with Ollama

Create a file named Modelfile:

FROM ./mimo-ornith-9b-agsi-abliterated-q4_k_m.gguf

PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.6
PARAMETER top_p 0.95

Build and execute the model:

ollama create mimo-ornith-9b-abliterated -f Modelfile
ollama run mimo-ornith-9b-abliterated

---

🛠️ Multi-Step Tool Calling (Hermes / Qwen XML Schema)

The GGUF template natively supports Hermes-style tool calling. When passing tool specifications via OpenAI-compatible payloads, the model produces:

<tool_call>
<function=bash>
<parameter=command>
python3 -c "import socket; print(socket.gethostname())"
</parameter>
</function>
</tool_call>

When passing back execution feedback, wrap the response as:

<|im_start|>user
<tool_response>
myserver.local
</tool_response><|im_end|>

---

💻 Python Client Example (OpenAI API Compatibility)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="none"
)

response = client.chat.completions.create(
    model="mimo-ornith-9b-agsi-abliterated",
    messages=[
        {"role": "user", "content": "Develop an automated python script to analyze memory fragmentation in Linux."}
    ],
    temperature=0.6,
    extra_body={
        "reasoning_effort": "high",
        "chat_template_kwargs": {
            "preserve_thinking": True
        }
    }
)

print(response.choices[0].message.content)

---

⚠️ Disclaimer

This model has been abliterated to eliminate refusal constraints for security auditing, systems programming, and technical problem-solving. Users are responsible for ensuring that outputs generated by this model comply with applicable local and international legal standards.