Agnes-3.0-Flash — MLX oQ4e + MTP (text-only, no vision tower)

oQ4e (mixed-precision) MLX quantization of Agnes-AI/Agnes-3.0-Flash (Apache-2.0), with the vision tower stripped out and Agnes's native MTP head kept for speculative decoding.

This is the text-only variant — no vision tower, ~18 GB on disk (vs ~20 GB with vision), same language model and MTP performance.

What changed from the vision version

• Vision tower removed: all 333 vision_tower.* tensors stripped from shard 5
• Shard 5 shrunk: from ~1.2 GB down to ~317 MB (only the 29 MTP tensors remain)
• vision_config kept in config.json: this is deliberate — it tells oMLX to route to VLMBatchedEngine, which is where MTP actually works for Qwen3.5 models. Strip the config too and MTP won't activate.
• MTP head intact: 29 tensors, same as the original — speculative decoding works identically
• Language model untouched: same 72 layers, same quantization, same everything else

Model Details

Performance

Measured on M3 Max 64GB, oMLX, temperature 0:

MTP gives roughly the same 1.4× speedup as the vision version. Removing the vision tower saves ~1.7 GB on disk and a bit of load time, but doesn't meaningfully change text inference speed.

Bit Distribution

Most layers at 4-bit, sensitive layers bumped to 5-bit, embedding at 8-bit:

Full per-layer breakdown is in oq_imatrix_report.json.

Usage

oMLX (recommended)

# Place the model directory in your oMLX models folder:
#   ~/.omlx/models/hermitdave/Agnes-3.0-Flash-qwen35-oQ4e-mtp-no-vision-V3/
# Restart oMLX — the model appears in the dropdown
# MTP is enabled by default, no extra config needed

Via the API:

curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Agnes-3.0-Flash-qwen35-oQ4e-mtp-no-vision-V3",
    "messages": [{"role": "user", "content": "Explain quantum computing in simple terms."}],
    "max_tokens": 200,
    "temperature": 0
  }'

Python (mlx_lm)

from mlx_lm import load, generate

model, tokenizer = load("hermitdave/Agnes-3.0-Flash-qwen35-oQ4e-mtp-no-vision-V3")

response = generate(
    model,
    tokenizer,
    prompt="The capital of France is",
    max_tokens=128
)
print(response)

Architecture

Agnes-3.0-Flash is a 33B hybrid attention model:

• 54 Gated DeltaNet (recurrent) layers + 18 global attention layers
• Repeats in blocks of 4: 3 DeltaNet → 1 global attention
• GQA 6:1: 24 query heads, 4 key/value heads, head dim 256
• 3-axis RoPE, causal conv kernel 4
• Hidden size 5120, intermediate size 19456
• MTP head: 1 layer, intermediate 17408 (Agnes native)

How this was made

Started from Agnes-3.0-Flash-qwen35-oQ4e-mtp (which has a vision tower grafted from Qwen3.8-27B). The process:

1. Copied the full model to a working directory
2. Opened shard 5 (model-00005-of-00005.safetensors) with safetensors
3. Read every tensor, dropped the 333 vision_tower.* ones, kept everything else (29 MTP tensors + any other non-vision tensors)
4. Wrote a new shard 5 with only the kept tensors
5. Updated model.safetensors.index.json to remove the 333 vision weight refs
6. Left vision_config and vision-related keys in config.json untouched — this is what makes oMLX route to VLMBatchedEngine so MTP works
7. Verified: all 5 shards match their index entries, VLMBatchedEngine loads it, MTP activates, text generation works

The key thing to understand: there are two ways to strip vision, and only one keeps MTP working. If you also strip vision_config from config.json, oMLX classifies the model as a pure LLM and routes to BatchedEngine — and MTP doesn't activate there. Keep the config, strip only the weights, and VLMBatchedEngine detects no vision tower at load time and falls back to text-only mode while still running MTP.

Index integrity

All 5 shards verified against their index entries:

Total: 2106 tensors. Every ref in model.safetensors.index.json matches an actual tensor in the corresponding shard.

Attribution

• Original model: Agnes-AI
• Vision tower (in original): Qwen3.8-27B
• Converted with Hermes Agent + mlx_lm
• Quantized with oMLX