Ornith-1.5-122B-EXL3-SAGE-2.58bpw

Quantized EXL3 pack of ornith-ai/Ornith-1.5-122B by vcruz305 with SAGE.

SAGE is our own mixed-precision quantization method for EXL3. The method is not published. If a tensor is EXL3, its K is self-describing on the trellis.

• Serving Recipe & Benchmarks Repository (NVIDIA DGX Spark / GB10 + vLLM 0.32.0 + vcruz305/vllm-exl3):
  👉 https://github.com/vcruz305/Ornith-1.5-122B-EXL3-DGX-Spark-recipe
• vLLM EXL3 Plugin:
  👉 https://github.com/vcruz305/vllm-exl3

---

Pack Summary

---

Measured Speeds on 1× NVIDIA DGX Spark (GB10, sm_121, 128 GB Unified)

Measured end-to-end with vLLM 0.32.0 + vcruz305/vllm-exl3 (using vcruz305/Ornith-1.5-122B-EXL3-DGX-Spark-recipe) and REXL3:

---

9-Suite (353/356-Item) Downstream Benchmark vs. 4.0 bpw

Measured on the same DGX Spark (GB10) under identical deterministic prompts and evaluation harnesses:

---

Fidelity vs. Unquantized ornith-ai/Ornith-1.5-122B BF16 Master

Measured directly against the unquantized ornith-ai/Ornith-1.5-122B BF16 master across held-out evaluation splits over the full 248,320-token vocabulary:

---

Quickstart: Serving with vLLM 0.32.0 + vcruz305/vllm-exl3

Full scripts, preflight checks, and benchmarks are available in the companion recipe repository:
👉 https://github.com/vcruz305/Ornith-1.5-122B-EXL3-DGX-Spark-recipe

git clone https://github.com/vcruz305/Ornith-1.5-122B-EXL3-DGX-Spark-recipe.git
cd Ornith-1.5-122B-EXL3-DGX-Spark-recipe

# 1. Create the vLLM environment (vllm==0.32.0, vcruz305/vllm-exl3 with -DP2B_CB=2, vision patch).
bash vllm-plugin/setup_env.sh

# 2. Download this pack.
HF_XET_HIGH_PERFORMANCE=1 ~/venvs/vllm-exl3/bin/hf download \
  vcruz305/Ornith-1.5-122B-EXL3-SAGE-2.58bpw \
  --local-dir ~/models/Ornith-1.5-122B-EXL3-SAGE-2.58bpw

# 3. Prepare a non-destructive symlinked view at ~/models/Ornith-1.5-122B-EXL3-SAGE-2.58bpw-vllm.
PACK_DIR=~/models/Ornith-1.5-122B-EXL3-SAGE-2.58bpw \
  PLUGIN_REPO=/path/to/vllm-exl3 \
  bash vllm-plugin/prepare_pack.sh

# 4. Launch vLLM (--quantization exl3, 262,144 context, VLLM_EXL3_COOP=1, VLLM_EXL3_COALESCE_SHARDS=1).
source ~/venvs/vllm-exl3/bin/activate
bash vllm-plugin/serve.sh

• Enable MTP speculative decoding (32.60 tok/s / 32.99 peak on code):

  SPEC_CONFIG='{"method":"mtp","num_speculative_tokens":1}' bash vllm-plugin/serve.sh
  

• Enable the 27-layer multimodal vision tower:

  LANGUAGE_MODEL_ONLY=0 bash vllm-plugin/serve.sh
  

---

About the Base Model: ornith-ai/Ornith-1.5-122B

(Copied from the upstream ornith-ai/Ornith-1.5-122B model card for reference)

Chirp Chirp! 🐦 We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement.

Ornith-1.5 extends Ornith-1.0 (which was developed on top of Qwen3.5 and Gemma4 with additional continued pretraining, mid-training, and post-training) by expanding the self-improvement loop from scaffold and rollout optimization to jointly optimizing task generation, scaffold construction, and solution rollouts. Rather than relying on a fixed set of human-curated tasks and manually designed harnesses, Ornith-1.5 continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning. For more details on the task, harness, and rollout reward design, please refer to our blog.

Upstream Benchmarks (ornith-ai/Ornith-1.5-122B)

Recommended Sampling & Reasoning Parameters

📝 NOTE

Ornith-1.5-122B is a reasoning model: by default the assistant turn opens with a <think> … </think> block before the final answer. Our vLLM recipe enables --reasoning-parser qwen3 so the chain-of-thought is returned in a separate reasoning_content field, and --tool-call-parser qwen3_xml --enable-auto-tool-choice so <tool_call> blocks are surfaced as standard OpenAI tool_calls.

Recommended sampling parameters:

• For general tasks: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0
• For precise coding tasks: temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0

Using Ornith-1.5-122B via the Chat Completions API

Once your vLLM server is running (http://localhost:8899/v1 via vcruz305/Ornith-1.5-122B-EXL3-DGX-Spark-recipe), talk to it with any OpenAI-compatible client:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8899/v1",
    api_key="EMPTY",
)

response = client.chat.completions.create(
    model="Ornith-1.5-122B-EXL3",
    messages=[
        {"role": "user", "content": "Write a one-line Python lambda that squares a number."}
    ],
    temperature=0.6,
    top_p=0.95,
    max_tokens=1024,
)

message = response.choices[0].message
print("reasoning:", getattr(message, "reasoning_content", None))
print("answer:", message.content)

Tool calling works out of the box:

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    }
]

response = client.chat.completions.create(
    model="Ornith-1.5-122B-EXL3",
    messages=[{"role": "user", "content": "What is the weather in Paris right now?"}],
    tools=tools,
    tool_choice="auto",
    temperature=0.6,
    max_tokens=2048,
)

tool_call = response.choices[0].message.tool_calls[0]
print(tool_call.function.name, tool_call.function.arguments)
# -> get_weather {"city": "Paris"}

Agentic & Coding CLI Usage

Hermes Agent

export OPENAI_BASE_URL="http://localhost:8899/v1"
export OPENAI_API_KEY="EMPTY"
export MODEL="Ornith-1.5-122B-EXL3"

OpenClaw

export OPENAI_BASE_URL="http://localhost:8899/v1"
export OPENAI_API_KEY="EMPTY"
export OPENAI_MODEL="Ornith-1.5-122B-EXL3"

OpenCode

# Register your local Ornith endpoint as a provider in ~/.config/opencode/opencode.json:
# {
#   "$schema": "https://opencode.ai/config.json",
#   "provider": {
#     "ornith": {
#       "npm": "@ai-sdk/openai-compatible",
#       "name": "Ornith (local)",
#       "options": { "baseURL": "http://localhost:8899/v1", "apiKey": "EMPTY" },
#       "models": { "Ornith-1.5-122B-EXL3": { "name": "Ornith-1.5-122B-EXL3" } }
#     }
#   }
# }
opencode

Citation

@misc{ornith_1_5,
    title = {{Ornith-1.5}: From Self-Scaffolding to Self-Improvement},
    url = {https://ornith.ai/ornith_1_5.html},
    author = {{Ornith Team}},
    year = {2026}
}