Ornith-1.5-9B for NInfer

This repository contains the registered NInfer artifact for Ornith-1.5-9B.

The artifact format (.ninfer) is designed specifically for NInfer's from-scratch C++/CUDA inference engine on NVIDIA Blackwell GPUs (verified on NVIDIA GeForce RTX 5060 Ti and RTX 5090). It is not a GGUF, AWQ, or standard Hugging Face Transformers checkpoint.

Note: Converted, tested, and verified autonomously by Hermes Agent × Codex Agent (gpt-5.6-luna).

---

Artifact Details

Verify the downloaded artifact:

printf '%s  %s\n' \
  'c465a06c9d32339493fd5000512724604b12965eb0a8963abe90acfc470f3fbe' \
  'ornith_1_5_9b.ninfer' | sha256sum --check

---

Performance on NVIDIA GeForce RTX 5060 Ti (16 GB)

• Long-Prompt Prefill Throughput: ~2,581 tok/s (with --prefill-chunk 4096 --kv-dtype int8).
• Concurrent MTP3 Serving Throughput (--spec mtp --draft-tokens 3 --lm-head-draft):
  • C = 1: ~108.2 tok/s (MTP acceptance ~60%, ~2.7 tok/round)
  • C = 4: ~191.1 tok/s
  • C = 8: ~309.9 tok/s aggregate (steady-state engine decode: ~339.6 tok/s)
• VRAM Footprint: ~5.52 GiB (model weights + MTP layers), leaving plenty of room for multi-client KV cache on 16GB GPUs.

---

Download & Run

Download

Using hf:

hf download ruwwww/ornith-1.5-9b-ninfer \
  ornith_1_5_9b.ninfer \
  --local-dir models

Run CLI

./build/apps/ninfer models/ornith_1_5_9b.ninfer \
  --prompt "Explain photosynthesis in three sentences." \
  --spec mtp --draft-tokens 3 \
  --lm-head-draft \
  --max-new 256

Run OpenAI-Compatible HTTP Server

./build/apps/ninfer-serve models/ornith_1_5_9b.ninfer \
  --host 0.0.0.0 --port 8080 \
  --max-concurrency 8 \
  --max-context 16384 \
  --kv-capacity auto \
  --spec mtp --draft-tokens 3 \
  --lm-head-draft

---

Provenance & Attribution

• Base Model: ornith-ai/Ornith-1.5-9B / ornith-ai/Ornith-1.5-9B-NVFP4
• Engine Repository: ruwwww/ninfer-5060ti
• Automation: Hermes Agent × Codex Agent (gpt-5.6-luna)