Qwythos-27B-v1-MXFP4-Vision-MTPLX

MLX/VLM conversion of empero-ai/Qwythos-27B-v1, pinned to revision 7c72a9c714cf66281cb222c4aa0aef368d84c94f.

Format

• Language body: MXFP4, 4-bit, group size 32
• Vision tower: source BF16, 333 tensors
• Architecture: dense Qwen3.5 multimodal, 64 language layers
• Chat template and tokenizer: preserved from the pinned source
• Packaging: self-contained; no companion model repository is required

Local qualification

• Structural inspection: pass (1,682 indexed tensors; 498 quantized weights; no stale or missing shards)
• Strict MLX-VLM load: pass
• Text behavior: 5/5 exact matches against BF16 for identity, JSON, code, tool calling, and loop tasks
• Vision smoke: pass; post-graft vision features are bit-for-bit equal to BF16
• Long-context needle: exact at 19,069 input tokens

The source configuration's 1M-token YaRN settings are preserved, but this release was locally exercised only through 19,069 input tokens—not at 1M.

MTPLX runtime

This variant packages the complete MXFP4 body and vision tower with the source model's native 15-tensor BF16 MTP head for MTPLX 2.0.1. Depth 2 is the recommended setting.

mtplx inspect --require-mtp <local-path-or-repo-id>
mtplx serve --model <local-path-or-repo-id> --generation-mode mtp --mtp-depth 2 --reasoning-mode off

Local 192-token tuning measured 44.96 decode tok/s at depth 2 versus 25.39 autoregressive (1.77x). In a deterministic 10-prompt API comparison, depth 2 and autoregressive outputs were byte-identical on all 10 prompts; both passed the same 6/10 validators. Mean decode throughput was 49.45 versus 27.87 tok/s. A separate code smoke passed 3/3. Thermal fan control remained on auto.

Basic use

python -m mlx_vlm.generate \
  --model <local-path-or-repo-id> \
  --image image.jpg \
  --prompt "Describe this image." \
  --max-tokens 256

Review the source model card for intended use, training, license, and deployment considerations. This conversion does not change the source model's safety characteristics.