Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ6-mtp
An oQ6 quantized MLX version of Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16-mtp with integrated Multi-Token Prediction (MTP) head for accelerated speculative decoding on Apple Silicon.
Quantized with oMLX oQ6 — a higher-precision mixed-quantization variant that preserves more layers at higher fidelity for better quality at the cost of increased memory usage compared to oQ4.
Model Summary
- Format: MLX safetensors, oQ6 quantized
- Precision: Mixed (oQ6 smart quantization)
- Parameters: ~9B total
- Context length: 1,048,576 tokens
- Architecture: Qwen3.5-style hybrid attention text model with MTP head
- MTP layers: 1 (embedded,
mtp_num_hidden_layers: 1) - Primary use: local text generation and reasoning on Apple Silicon with MTP-accelerated decoding
- Base model:
xunkutech-ai/Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16-mtp
Compatibility Notice
oMLX is required to load this model with MTP support. Standard
mlx-vlm,mlx-lm, and LM Studio do NOT support MTP and will fail with:ValueError: Received 15 parameters not in model: language_model.mtp.*.
Installation
pip install -U git+https://github.com/jundot/omlx.git
Usage with oMLX
# Start server with MTP draft-mode
omlx serve --model /path/to/Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ6-mtp
from omlx import load, generate
model_id = "/path/to/Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ6-mtp"
model, tokenizer = load(model_id)
result = generate(
model=model,
tokenizer=tokenizer,
prompt="Explain speculative decoding with MTP.",
max_tokens=2048,
temperature=0.6,
)
print(result)
Files
| File | Size | Description |
|---|---|---|
config.json |
~12 KB | oQ6 quantized config with mtp_num_hidden_layers: 1 |
model-00001-of-00002.safetensors |
~5.0 GB | Quantized shard 1/2 |
model-00002-of-00002.safetensors |
~3.5 GB | Quantized shard 2/2 (+ MTP head) |
tokenizer.json |
~20 MB | Qwen3.5 tokenizer |
chat_template.jinja |
~8 KB | Qwythos chat template |
Recommended Sampling
generation_kwargs = {
"temperature": 0.6,
"top_p": 0.95,
"top_k": 20,
"repetition_penalty": 1.05,
"max_tokens": 4096,
}
License
Released under the same license as the upstream model: Apache-2.0.
Acknowledgements
- Original model: empero-ai/Qwythos-9B-Claude-Mythos-5-1M
- Original developers: Empero AI
- Base family: Qwen3.5
- MTP support: oMLX
- MLX ecosystem: Apple MLX