How to Run Ornith 1.5 9B Locally

Built from Ornith's original weights with our own importance matrix. The calibration corpora and raw eval logs behind this card are public.

Learn how to run Ornith 1.5 9B locally - read our guide.

You can now run Ornith 1.5 in Atomic Chat with toggles for thinking.

See our quantization analysis below for measurements and instructions.

Ornith-1.5-9B-GGUF

GGUF builds of ornith-ai/Ornith-1.5-9B,

converted and measured by AtomicChat.

Which file to take

Sizes below already include the 0.9 GB vision projector and about 1 GB of

runtime overhead at 8k context. Drop the projector if you only want text.

| Your VRAM | Take | Why |

|---|---|---|

| 24 GB and up | BF16 | The reference itself. Nothing to lose. |

| 16 GB | Q8_0 | Stock, and stock is already optimal this high up. |

| 12 GB | AD-Q8_0-Q6_K | Fills the gap between Q6_K and Q8_0. |

| 8 GB | AD-Q5_K-Q4_K | The sweet spot. Smaller than stock Q5_K_M and more accurate. |

| 8 GB, tight | AD-Q4_K-IQ4_XS | Same size as stock Q4_K_M, 31% lower divergence. |

| 6 GB | AD-IQ3_S-IQ3_XXS | Usable, but top-1 agreement drops to 83%. |

| 4 GB | AD-IQ2_S-IQ2_XS, text only | 71% top-1. It runs. Read the warning below. |

[!WARNING]

The two smallest files are published for completeness, not because we

recommend them. AD-IQ2_XXS-IQ1_M agrees with the reference on 53.7% of

tokens, which means roughly every second token is a different word than the

full model would have produced. That is not a small quality drop, it is a

different model. If a 9B does not fit, a smaller model at higher precision

will serve you better.

Files and measurements

Mean KL divergence against the BF16 reference, held-out eval_neutral corpus,

4096 context, single RTX 5090, llama.cpp b10505. Lower is better. Top-1 is the

share of tokens where the quant picks the same next token as BF16.

|---|---|---|---|

| Ornith-1.5-9B-BF16.gguf | 17.9 GB | reference | 100% |

| Ornith-1.5-9B-Q8_0.gguf | 9.53 GB | 0.002249 | 97.94% |

| Ornith-1.5-9B-AD-Q8_0-Q6_K.gguf | 8.55 GB | 0.003473 | 97.46% |

| Ornith-1.5-9B-Q6_K.gguf | 7.36 GB | 0.006045 | 96.54% |

| Ornith-1.5-9B-Q5_K_M.gguf | 6.47 GB | 0.029883 | 92.80% |

| Ornith-1.5-9B-AD-Q5_K-Q4_K.gguf | 5.93 GB | 0.025493 | 93.10% |

| Ornith-1.5-9B-AD-Q4_K-IQ4_XS.gguf | 5.61 GB | 0.034426 | 91.93% |

| Ornith-1.5-9B-AD-IQ4_XS.gguf | 5.52 GB | 0.037752 | 91.54% |

| Ornith-1.5-9B-IQ4_XS.gguf | 5.20 GB | 0.044856 | 90.76% |

| Ornith-1.5-9B-IQ3_M.gguf | 4.42 GB | 0.131672 | 84.18% |

| Ornith-1.5-9B-AD-IQ3_S-IQ3_XXS.gguf | 4.29 GB | 0.144132 | 83.44% |

| Ornith-1.5-9B-AD-IQ3_XXS-IQ2_S.gguf | 3.84 GB | 0.258554 | 77.98% |

| Ornith-1.5-9B-AD-IQ2_S-IQ2_XS.gguf | 3.38 GB | 0.441580 | 71.17% |

| Ornith-1.5-9B-AD-IQ2_XXS-IQ1_M.gguf | 2.81 GB | 1.122010 | 53.74% |

AD means a per tensor bit layout tuned for this model. Files without the

prefix are stock llama.cpp presets built with the same importance matrix, so the

comparison isolates the layout and nothing else.

!image_2026-08-19_18-00-17

Two results are worth reading off that chart directly.

**AD-Q5_K-Q4_K is smaller than stock Q5_K_M and more accurate at the same

time**, 5.93 GB against 6.47 GB and 0.0255 against 0.0299. No interpolation

needed to see it.

At the same size, the tuned layout cuts divergence by about a third.

AD-Q4_K-IQ4_XS is 5.61 GB at 0.0344 against stock Q4_K_M at 5.63 GB and

0.0500.

[!NOTE]

The tuning only pays while the base type is coarse. Above six bits the stock

presets are already close to lossless, so lifting a small part of the model

buys nothing while paying for it out of a large part costs real accuracy. We

tested this and lost, so above 6 GB this repo ships stock presets rather than

a worse tuned file with our label on it.

Quick start

Text only:

llama-server -m Ornith-1.5-9B-AD-Q5_K-Q4_K.gguf \
  -ngl 99 -c 8192 -fa on --jinja \
  --temp 0.6 --top-p 0.95 --top-k 20 \
  --port 8080

With vision:

llama-mtmd-cli -m Ornith-1.5-9B-AD-Q5_K-Q4_K.gguf \
  --mmproj mmproj-Ornith-1.5-9B-F16.gguf \
  -ngl 99 -c 8192 --temp 0.6 --image-min-tokens 1024 \
  --image your-image.jpg \
  -p "Describe this image."

[!IMPORTANT]

Pass --image-min-tokens 1024. Without it, dense images such as charts and

screenshots get too few visual tokens, and the model answers from what it

already knows instead of from what it was shown. llama.cpp prints a warning

about this at load time. We hit exactly this on our first attempt: the model

produced a confident, fluent, completely invented description of a chart.

[!IMPORTANT]

Upstream ships no generation_config.json, so llama.cpp falls back to its own

sampling defaults, which are not the ones this model was tuned for. Set them

yourself: --temp 0.6 --top-p 0.95 --top-k 20. Use --temp 1.0 to reproduce

the benchmark setup from the upstream model card.

Lines reading find_slot: non-consecutive token position are normal. That is

how this family numbers image patches, and they appear in successful runs too.

Speed

Single RTX 5090, llama.cpp b10505-ee4c505a4, CUDA 13.0, flash attention on.

| Test | BF16 |

|---|---|

| Prompt processing, 512 tokens | 9047 t/s |

| Generation, 128 tokens | 94.3 t/s |

Generation is limited by memory bandwidth, not compute: 17.9 GB read per token

against 1792 GB/s puts the ceiling near 100 t/s, and BF16 reaches 94% of it.

Smaller files scale close to linearly, so a 5.9 GB build lands near 280 t/s on

the same card.

Vision

The model reads handwriting. Asked to describe a calligraphy piece it has not

seen before, it transcribed the full quote and picked out the small TB-09 mark

in the corner.

!input image

!model output

This image displays an elegant, calligraphic handwritten quote featuring a

large blue initial capital "P" and intricate botanical illustrations in the

corners. The text reads: "Peace of mind comes to me through making things

with my hands. An added bonus comes if my efforts inspire others to try the

creative process."

Image encoding takes 163 ms for this picture on one RTX 5090.

Reasoning

A short check that the thinking channel does real work rather than filling

space. Prompt: *A gas station is 1 kilometer away. I want to wash my car. Should

I walk or drive?*

!thinking trace

The model answered "drive", then argued against itself in the open, noting that

some gas stations have car washes so walking there is not obviously wrong, then

closed the argument by observing that walking leaves the car at home either way:

You should drive. To wash your car, your car needs to be at the location

where it is being washed.

How the layout was chosen

The bit layout was not carried over from another model. We built eight candidate

layouts at one size class, measured all of them against the same reference on

the same box, and kept the winner. Every log below is in the metrics repo.

|---|---|---|---|---|

| AB-G | 5.58 GB | 0.034841 | 91.85% | attn_gate and ssm_out up, MLP gate and up down |

| AB-Q | 5.71 GB | 0.044434 | 91.04% | split attn_q: full attention layers up, linear layers down |

| AB-E | 5.64 GB | 0.045923 | 90.93% | followed the energy bands from the importance matrix |

| AB-U | 5.57 GB | 0.048779 | 90.60% | control, flat across layers |

| AB-S | 5.61 GB | 0.049737 | 90.50% | ssm_in up |

| Q4_K_M | 5.63 GB | 0.049981 | 90.47% | stock preset, same importance matrix |

| AB-X | 5.62 GB | 0.051900 | 90.45% | edge layers lifted, a recipe that won on another model |

| AB-T | 5.67 GB | 0.066871 | 89.21% | embedding down hard, output head up hard |

Three things came out of this.

The gates matter more than their size suggests. attn_gate and ssm_out

are 9% of the weights, and the importance matrix ranks attn_gate first in the

whole model by summed squared activation. Lifting those two and paying for them

out of the MLP gate and up projections, which are 36% of the weights, is the

single largest win available.

The embedding must not be squeezed. With a 248320 token vocabulary and

untied weights, the embedding and the output head are 22% of this model, which

makes them look like the obvious place to save. AB-T tested that and came last,

worse than the stock preset. A lifted output head does not pay for a crushed

embedding.

A layout does not transfer by layer position. AB-X applied a recipe that

works well on a 64 layer model by scaling the layer numbers, and it lost to the

flat control. What transfers is which tensor groups matter, read from the

importance matrix of this model.

Architecture

Read from the converted file, not from the config.

| | |

|---|---|

| Parameters | 8.95B language, 0.46B vision |

| Layers | 32 |

| Attention | hybrid: full attention on layers 3, 7, 11, 15, 19, 23, 27, 31, linear attention on the other 24 |

| Hidden size | 4096 |

| FFN size | 12288 |

| Vocabulary | 248320, untied embeddings |

| Context | 262144 |

| Architecture string | qwen35 |

Only 8 of the 32 layers keep a KV cache, the other 24 hold a fixed recurrent

state instead. Long context is unusually cheap here, roughly 32 KB per token, so

8k of context costs about 256 MB rather than the gigabytes this size of model

normally implies.

[!WARNING]

If you convert this model yourself, pass --no-nextn to

convert_hf_to_gguf.py. Upstream config.json declares

mtp_num_hidden_layers: 1, but the checkpoint contains no such tensors. The

converter believes the config, writes block_count = 33 for a 32 layer model,

and reports a successful export. The file then fails to load with

tensor 'blk.32.attn_norm.weight' not found. There is no speculative draft to

publish for this model.

Calibration

The importance matrix was computed on the BF16 weights, not on a quantized

stand-in, over 4,958,490 tokens in 9,686 chunks, fanned out across 20 GPUs and

merged. Vocabulary coverage of the calibration corpus is 99.5%, so almost no

embedding row was quantized without evidence.

Corpus composition: agentic tool traces 24.7%, code 17.8%, reasoning 14.8%,

multilingual 13.8%, long context 11.9%, vocabulary sweep 9.9%, structured data

3.9%, graphics 3.0%. The vocabulary sweep is regenerated per tokenizer, since a

sweep built for another model covers a different vocabulary and calibrates

nothing here.

Reproducing our numbers

Everything needed to check this card is public:

The BF16 file in this repo is the reference every measurement is taken against.
The calibration corpus is AtomicChat/calib-corpora, build ornith-1.5-9b.
Ornith-1.5-9B-GGUF-metrics

holds the importance matrix, every raw KL divergence log including the losing

ablations, and the reference logits themselves, so you can measure your own

build against the same starting point instead of taking our word for it.

To measure a build of your own:

llama-perplexity -m your-quant.gguf -f eval_neutral.txt \
  --kl-divergence-base base-neutral.kld --kl-divergence -c 4096 -ngl 99

The corpus and the context have to match ours or the numbers are not comparable.

About the model

Ornith-1.5 is trained with a loop that generates its own tasks, builds its own

scaffolds, and rewards all three stages together rather than optimizing a policy

against a fixed human written harness. For the method and the benchmark results,

see the upstream model card

and the Ornith blog. Those benchmark numbers are the

authors' own and we have not independently reproduced them.

AtomicChat/Ornith-1.5-9B-GGUF