Qwen3.8-27B-OBLITERATED-V3-Sharp-oQ4e-mtp (Multimodal Merged & Restored Build)

This is a Multimodal Merged & Restored Release based on eternalis/Qwen3.8-27B-OBLITERATED-V3-Sharp-oQ4e-mtp and OBLITERATUS/Qwen3.8-27B-OBLITERATED.

This release performs a cross-precision multimodal architecture merge:

1. Multimodal Architectural Stitching: Re-integrates the full 27-layer unquantized BF16 ViT Vision Tower (vision_tower.safetensors, 879 MB, 333 visual tensors) that was stripped in upstream text-only releases, fully restoring high-resolution visual comprehension and OCR capabilities.
2. Language Backbone & Speculative Head: Marries the direction-ablated 4-bit (oQ4e) language backbone from eternalis with its memory-bandwidth-optimized 4-bit MTP draft head, achieving 34 ~ 41.5 tok/s output on Apple Silicon.
3. Full Multimodal Pipeline: Includes complete preprocessor configurations, spatial/temporal projection alignments, and vision token mappings for plug-and-play inference in omlx and mlx-vlm.

---

🌟 Key Highlights & Enhancements

1. 👁️ Full Precision Vision Tower Restored (BF16 ViT)

   • The original eternalis release removed all vision components to save disk space, rendering the model text-only.
   • This release restores the complete unquantized BF16 ViT Vision Tower (vision_tower.safetensors, 879 MB, 333 visual tensors, 27 layers).
   • Retaining unquantized BF16 for the vision tower prevents quantization noise, ensuring pixel-perfect OCR, fine diagram reading, and high-accuracy document comprehension.
   • During text-only generation, the vision tower remains idle with 0% memory bandwidth overhead.

2. ⚡ Native 4-bit MTP Speculative Decoding (depth=3)

   • Includes the 4-bit Multi-Token Prediction draft head.
   • Extensively benchmarked on Apple Silicon (M5 Pro):
     • Generation speed: 34 ~ 41.5 tok/s (peaking at 41.5 tok/s on reasoning tasks).
     • Prefill speed: ~360 tok/s for multimodal image + text prefill.
     • MTP Acceptance Rate: ~74.6%.

3. 🛠️ Complete Multimodal Processing Pipeline

   • Fully patched config.json with vision_config (spatial merge size: 2, temporal patch size: 2, depth: 27, hidden size: 1152).
   • Vision token IDs registered (<|vision_start|>, <|vision_end|>, <|image_pad|>).
   • Includes preprocessor_config.json, processor_config.json, and video_preprocessor_config.json.
   • Preserves the Qwen Sharp Chat Template for concise responses.

---

📊 Architecture & Specifications

---

🚀 Usage

1. With oMLX Server (Recommended)

Place this model into your ~/.omlx/models/ directory or run:

omlx serve song2boy/Qwen3.8-27B-OBLITERATED-V3-Sharp-oQ4e-mtp --port 8000

oMLX will automatically detect both vlm engine and native MTP speculative decoding.

2. With mlx-vlm CLI

mlx_vlm.generate \
  --model song2boy/Qwen3.8-27B-OBLITERATED-V3-Sharp-oQ4e-mtp \
  --image /path/to/your/image.png \
  --prompt "Describe this image in detail." \
  --max-tokens 512

---

📜 Credits & Attribution

• Original Abliteration & weights by OBLITERATUS.
• MLX oQ4e text-only quantization by eternalis.
• Qwen Sharp Chat Template by peculiar-ragdoll.
• Multimodal architecture merge, cross-precision pipeline fusion, and benchmarking by song2boy.