Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4-MTP-GGUF

Experimental GGUF conversion of AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4 with an embedded native Qwen3.6 MTP block.

This is a self-contained NVFP4 GGUF with embedded native MTP tensors.

It does not require an external MTP-only draft model, but this embedded-MTP path is experimental.

Status

Experimental.

The model loads and performs native MTP speculative decoding in llama.cpp. However, on the tested system, using the same MTP tensors through a standalone MTP-only GGUF via --model-draft was slightly faster than embedding the tensors directly into the GGUF.

For most users, the standard NVFP4 GGUF plus external --model-draft is recommended.

Model Details

• Source model: AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4
• Base model: deepreinforce-ai/Ornith-1.0-35B
• Format: GGUF
• Quantization: NVIDIA NVFP4
• Architecture: Qwen3.6 Mixture-of-Experts with hybrid attention
• MTP: Embedded native Qwen3.6 MTP block
• Parameters: 35B total, approximately 3B activated per token
• Purpose: Local inference and native MTP speculative decoding with llama.cpp

Embedded MTP

This model was created by grafting the native Qwen3.6 MTP block into the converted NVFP4 GGUF.

The resulting GGUF contains:

qwen35moe.block_count = 41
qwen35moe.nextn_predict_layers = 1

Additional tensors:

blk.40.*

Only the native MTP block was added. The original transformer layers blk.0.* through blk.39.* were not modified.

Recommended graft source:

Qwen3.6-35B-A3B-MTP-ONLY-Q6_K.gguf

Compatibility

A recent version of llama.cpp with Qwen3.6 MoE, NVFP4, and native MTP support is required.

Tested with:

• Windows
• NVIDIA GeForce RTX 5070 Ti 16 GB
• NVIDIA GeForce RTX 5060 Ti 16 GB
• llama.cpp CUDA backend
• Embedded native Qwen3.6 MTP speculative decoding

Older llama.cpp builds may fail to recognize the nvfp4 tensor type, may not correctly load associated scale tensors, or may lack compatible Qwen3.6 MoE/MTP support.

Performance may vary with llama.cpp build, GPU split, context size, KV-cache format, prompt, sampling settings, and other runtime options.

Usage

llama-server

llama-server \
  -m Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4-MTP.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --flash-attn on \
  -ngl 99

llama-cli

llama-cli \
  -m Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4-MTP.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --flash-attn on \
  -ngl 99

Multi-GPU example

llama-server \
  -m Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4-MTP.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --split-mode layer \
  --tensor-split 1,1 \
  --flash-attn on \
  -ngl 99

The best tensor split depends on available VRAM, GPU speed, PCIe topology, context length, and KV-cache placement. An even split is only a starting point.

Runtime Recommendation

Although the embedded MTP GGUF functions correctly, local benchmarks showed that using the standalone MTP-only GGUF via --model-draft was slightly faster than embedding the same tensors directly into the GGUF.

Recommended for most users:

llama-server \
  -m Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4.gguf \
  --model-draft Qwen3.6-35B-A3B-MTP-ONLY-Q6_K.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3

Use this embedded-MTP GGUF mainly if you want a self-contained file or want to experiment with native MTP grafting.

Benchmark Summary

Local mixed-task benchmark:

The embedded model works correctly, but --model-draft currently appears to be the better runtime path on the tested setup.

Results may vary depending on llama.cpp version, CUDA graph behavior, GPU offload, batch/ubatch settings, context length, quantization, and prompts.

Suggested MTP Settings

A reasonable general-purpose starting point is:

--spec-draft-n-max 3

For translation, role-play, creative writing, conversational output, or open-ended explanations, start with:

--spec-draft-n-max 2

For JSON, fixed templates, repeated patterns, and deterministic code completion, try:

--spec-draft-n-max 4

Grafting Method

The embedded MTP block was generated by grafting the standalone MTP-only GGUF.

This demonstrates that MTP-only GGUF files can serve two purposes:

1. Standalone draft model via --model-draft
2. Graft source for creating a self-contained GGUF

The graft process updates:

qwen35moe.block_count: 40 -> 41
qwen35moe.nextn_predict_layers: 1

and appends:

blk.40.*

It does not modify:

tokenizer
vocabulary
chat template
RoPE settings
original transformer layers

Verification

Check the embedded MTP metadata and tensors:

python .\gguf-py\gguf\scripts\gguf_dump.py Qwen3.6-35B-A3B-NVFP4-MTP.gguf |
  Select-String "block_count|nextn_predict_layers|blk\.40"

Expected output should include:

qwen35moe.block_count = 41
qwen35moe.nextn_predict_layers = 1
blk.40.attn_k.weight
blk.40.nextn.eh_proj.weight
blk.40.nextn.shared_head_norm.weight

Notes

• This GGUF preserves NVIDIA's NVFP4 tensor type; it is not equivalent to Q4_K_M, Q4_K_S, or an Unsloth dynamic quant.
• The embedded MTP block is experimental and may not be faster than using --model-draft.
• The model is a 35B-total-parameter MoE with approximately 3B parameters activated per token; runtime memory requirements still depend on the full stored checkpoint rather than only the active parameter count.
• NVIDIA's source checkpoint was prepared for ModelOpt and vLLM. llama.cpp support is a separate community implementation and may behave differently from NVIDIA's reference runtime.
• Native MTP accelerates token generation but does not improve prompt-prefill speed in the same way.
• Very short benchmark outputs are more sensitive to run-to-run variance.
• Results are specific to the tested hardware, llama.cpp build, prompts, runtime options, and context configuration.
• Although standard decoding completed slightly faster on this short benchmark, native MTP enables speculative decoding and substantially increases token generation throughput (up to ~115 tok/s in these tests). Acceptance rate is therefore a more meaningful metric than total wall time when evaluating MTP effectiveness.

Related Projects

• Standard NVFP4 GGUF: AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4
• Grafting utility: gguf-graft-mtp

Credits

• Qwen Team / Alibaba Cloud — Qwen3.6-35B-A3B
• deepreinforce-ai / Ornith-1.0-35B — Qwen3.6-35B-A3B
• NVIDIA — ModelOpt and the original NVFP4 checkpoint
• ggml-org — llama.cpp, GGUF, NVFP4 inference support, and native MTP support
• a4lg — MTP-only GGUF subset used as a graft source

License

The source model is distributed under the Apache License 2.0.

Users should review the upstream AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-NVFP4, deepreinforce-ai/Ornith-1.0-35B, and the MTP-only GGUF source model cards before redistribution or commercial use.