Ornith-1.5-122B-MTP-GGUF
Ornith-1.5-122B is a 122-billion-parameter Mixture-of-Experts (MoE) large language model developed by Ornith AI, released as an open-weight model under the MIT license and designed to deliver strong performance while remaining efficient enough to run on a single 80GB GPU (such as a DGX node), with an additional quantized "Mobile" variant for edge deployment. It functions as a reasoning model, generating an internal chain-of-thought within tags before producing its final answer, and supports agentic workflows through native tool/function calling compatible with OpenAI-style APIs. The model supports a native context window of up to 262,144 tokens, extendable to roughly 1M tokens via YaRN-based RoPE scaling, and it achieves state-of-the-art results among open-source models of comparable size across coding (e.g., SWE-bench Verified/Pro/Multilingual, TerminalBench, CyberGym, NL2Repo), reasoning (HLE, SciCode), and agentic benchmarks (WideSearch, BrowseComp, MCP-Atlas, Toolathlon, ClawEval), often matching or surpassing larger models like DeepSeek-V4-Flash and Beam. It integrates with modern serving frameworks (vLLM, SGLang, Transformers), GGUF-based runtimes (llama.cpp, Ollama), and popular agent/coding tools (OpenCode, Hermes Agent, OpenClaw, Unsloth), making it well-suited for coding assistance, complex task decomposition, and real-world agentic applications.
[!NOTE] MTP — Speculative Decoding: Yes
[!NOTE] Multi-Token Prediction (MTP) GGUF is a specialized GGUF model file format extension that integrates speculative decoding directly into the model weights to significantly accelerate local inference. Unlike traditional speculative decoding which requires a separate, smaller "draft" model, MTP GGUF files include additional output heads within the main model architecture that predict multiple future tokens in a single forward pass.
Model Files
| File Name | Quant Type | File Size | File Link | Description |
|---|---|---|---|---|
| Ornith-1.5-122B.BF16-00001-of-00006.gguf | BF16 (Part 1) | 48.6 GB | Link | Full BF16 weights (Part 1 of 6). Highest quality. |
| Ornith-1.5-122B.BF16-00002-of-00006.gguf | BF16 (Part 2) | 48.4 GB | Link | Full BF16 weights (Part 2 of 6). Highest quality. |
| Ornith-1.5-122B.BF16-00003-of-00006.gguf | BF16 (Part 3) | 48.6 GB | Link | Full BF16 weights (Part 3 of 6). Highest quality. |
| Ornith-1.5-122B.BF16-00004-of-00006.gguf | BF16 (Part 4) | 48.6 GB | Link | Full BF16 weights (Part 4 of 6). Highest quality. |
| Ornith-1.5-122B.BF16-00005-of-00006.gguf | BF16 (Part 5) | 48.6 GB | Link | Full BF16 weights (Part 5 of 6). Highest quality. |
| Ornith-1.5-122B.BF16-00006-of-00006.gguf | BF16 (Part 6) | 6.4 GB | Link | Full BF16 weights (Part 6 of 6). Highest quality. |
| Ornith-1.5-122B.Q3_K_L.gguf | Q3_K_L | 64.7 GB | Link | Lower quality but usable, good for low RAM availability. |
| Ornith-1.5-122B.Q4_K_M.gguf | Q4_K_M | 75.8 GB | Link | Good quality, default size for most use cases, recommended. |
| Ornith-1.5-122B.Q5_K_M.gguf | Q5_K_M | 88.7 GB | Link | High quality, recommended. |
| Ornith-1.5-122B.mmproj-bf16.gguf | mmproj-bf16 | 912 MB | Link | Multimodal projection file in BF16 format. Used for vision/language models. |
llama.cpp
LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp
Running Models with llama.cpp (llama-cli)
This guide outlines how to compile the llama-cli binary and run inference using Multi-Token Prediction (MTP) speculative decoding and KV cache quantization.
1. Build llama-cli
Navigate to your llama.cpp repository and compile the llama-cli target in Release mode:
cd /path/to/llama.cpp
cmake -B build
cmake --build build --config Release -j$(nproc) --target llama-cli
2. Run Inference
Execute the llama-cli binary with your desired GGUF model and runtime flags:
./build/bin/llama-cli \
-m /path/to/models/your-model.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 2 \
-ngl 99 \
-c 8192 \
--cache-type-k q8_0 \
--seed 42 \
-n 512 \
-st \
-p "Your prompt here"
Quantized using llama.cpp by prithivMLmods