Ornith-1.0-35B-FP8-BLOCK

This repository contains a public FP8_BLOCK / compressed-tensors quantization of deepreinforce-ai/Ornith-1.0-35B.

The original model is Ornith-1.0-35B, a Qwen3.5 MoE-family reasoning and agentic coding model released by DeepReinforce. This repository only changes the checkpoint representation; it does not introduce additional fine-tuning.

This model is a faithful quant of the original Ornith-1.0-35B and therefore does not have an MTP head, however, we have also uploaded a shisa-ai/Ornith-1.0-35B-FP8-BLOCK-MTP model that grafts the Qwen 3.6 35B-A3B official MTP head onto the model. This is the best-performing MTP and based on our testing it gives a >20% throughput uplift for coding workloads (tested on RTX PRO 6000 and vLLM v0.23.0).

Quantization Summary

• Source model: deepreinforce-ai/Ornith-1.0-35B
• Quantized model: shisa-ai/Ornith-1.0-35B-FP8-BLOCK
• Quantization tool: llm-compressor model-free PTQ
• Quantization format: compressed-tensors
• Scheme: FP8_BLOCK
• Calibration data: none; this is data-free/model-free PTQ
• Weight quantization: static FP8, symmetric, block strategy, 128x128 blocks
• Activation quantization: dynamic FP8, symmetric, group strategy, group size 128
• Target modules: Linear
• compressed-tensors metadata version recorded in config.json: 0.15.1.a20260406

The local quantization environment used for this artifact reported:

• llmcompressor=0.10.1.dev67+ga1cec6fa
• compressed-tensors=0.15.1a20260406
• transformers=5.5.0

The definitive machine-readable quantization metadata is in config.json.

Unquantized / Ignored Paths

The quantization run intentionally skipped the following module patterns:

re:.*lm_head$
re:.*embed_tokens$
re:.*visual.*
re:.*mlp\.gate$
re:.*mlp\.shared_expert_gate$
re:.*linear_attn.*
re:^mtp.*

Practical implications:

• lm_head and token embeddings remain unquantized.
• MoE router/gating paths remain unquantized.
• Qwen3.5 linear-attention/Gated-DeltaNet paths remain unquantized.
• Vision tower weights remain unquantized in this artifact.
• No MTP tensors were present in the source checkpoint used here, so this model card does not advertise MTP/speculative decoding support.

Example Usage

Use a recent runtime that supports both Qwen3_5MoeForConditionalGeneration and compressed-tensors FP8_BLOCK checkpoints.

vLLM

vllm serve shisa-ai/Ornith-1.0-35B-FP8-BLOCK \
  --served-model-name Ornith-1.0-35B-FP8-BLOCK \
  --tensor-parallel-size 8 \
  --host 0.0.0.0 \
  --port 8000 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.90 \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --trust-remote-code

Adjust --tensor-parallel-size, --max-model-len, and memory settings for your hardware. Long-context serving still requires substantial KV-cache memory even though the checkpoint weights are compressed.

OpenAI-Compatible Client

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="EMPTY",
)

response = client.chat.completions.create(
    model="Ornith-1.0-35B-FP8-BLOCK",
    messages=[
        {"role": "user", "content": "Write a Python function is_prime(n). Keep it short."}
    ],
    temperature=0.6,
    top_p=0.95,
    max_tokens=1024,
)

message = response.choices[0].message
print("reasoning:", getattr(message, "reasoning_content", None))
print("answer:", message.content)

Ornith is a reasoning model. With vLLM's --reasoning-parser qwen3, reasoning tokens are surfaced separately as reasoning_content; final answers remain in content.

Quality and Performance Notes

This upload documents the quantized checkpoint format and provenance. It does not publish a new retained benchmark table for the quantized model. For production use, validate quality and end-to-end serving latency against the BF16 source model on your target workload and runtime.

For original source-model capabilities, benchmark descriptions, and training context, see the upstream card: deepreinforce-ai/Ornith-1.0-35B.

License and Attribution

The source model is MIT licensed. This derivative quantized artifact keeps the source license metadata and links to the upstream license file.

If you use the source model, cite the original Ornith release:

@misc{ornith-35b,
    title = {{Ornith-1.0-35B}: Agentic Coding, Open to All},
    url = {https://deep-reinforce.com/ornith_1_0.html},
    author = {{DeepReinforce Team}},
    year = {2026}
}