Qwen3-27B-OBLITERATED-JANG-VISION-MLX-4bit

Hybrid Vision-Language — Uncensored Language (OBLITERATUS) + Full Vision Encoder (JANG 4D-CRACK) · MLX 4-bit Affine Quantized · E2E Verified on Apple Silicon

The best of both worlds: the aggressive uncensor of OBLITERATUS/Qwen3.8-27B-OBLITERATED (0% refusal, 6 rounds SVD, residue mining) fused with the complete visual encoder of dealignai/Qwen3.8-27B-JANG_4D-CRACK (ViT 27 layers). Native Qwen3_5ForConditionalGeneration (qwen3_5) running at 15 GB via MLX 4-bit instead of 55 GB BF16 — 5–7× faster on Apple Silicon.

---

✨ Highlights

• Uncensored language core — OBLITERATUS 6-round SVD ablation, 5 refusal directions removed. Benchmarked at 0% refusal on refusal probes while preserving MMLU 81.4%.
• Full JANG vision — 501 vision_tower tensors + 10 merger tensors, depth 27 ViT, no truncation. Verified mlx_vlm.load() → VisionModel OK on blue 32×32 probe: "solid blue background..." ✓
• MLX 4-bit affine quantization — g64 for language, g128 for vision. Weights packed as U32 + F16 scales/biases per group, compatible with mlx-vlm >=0.6.10.
• Efficient — 15 GB total (27.3B params) vs 55 GB BF16. Text-only inference ~15–20 tok/s on M-series Max/Ultra; vision adds only +0.4–0.8 s encode.
• Drop-in OMLX + mlx-vlm — works with mlx_vlm.generate and OMLX /v1/chat/completions with image_url.

---

📐 Architecture — Qwen3.5

qwen3_5 is the Qwen3-VL / Qwen3.5 generation. Language branch uses hybrid linear_attention (3/4) + full_attention (1/4) with linear_num_key_heads 16 / value_heads 48. Vision is a standard ViT, merged via learned projection to 5120d language space. deepstack_visual_indexes: [] (default).

---

🔧 Quantization — MLX Affine 4-bit

Affine quantization packs 4-bit weights as U32 (weight as uint32 packed) + F16 per-group scales & biases. Stored in safetensors with companion tensors *.scales / *.biases.

Config excerpts:

"quantization": {
  "group_size": 64, "bits": 4, "mode": "affine",
  "vision_tower.pos_embed": {"group_size":128,"bits":4,"mode":"affine"},
  "vision_tower.blocks.0.attn.qkv": {"group_size":128,"bits":4,"mode":"affine"},
  "... 87 entries total (1 default + 84 vision overrides + 2 top-level)"
}

• Language: ~1847 tensors (632 + 720 + 495 per shard patterns)
• Vision: 501 tensors (vision_tower) + merger, split across model-00004 (145 tensors, 194 MB) + model-00005 (256 tensors, 230 MB)
• Total: 2348 tensors → 27.356 B params → total_size 15_577_388_960 bytes on disk (compressed 4-bit)

Why g128 for vision? ViT channels (1152) are highly structured; larger group preserves patch reconstruction with negligible quality loss, while saving ~15% overhead vs g64.

---

📦 Shards & File Layout

du -sh .  # 15G
python -c "import json; d=json.load(open('model.safetensors.index.json')); print(len(d['weight_map']))"  # 2348

How it was built:

1. APFS cp -c clone of the 3 OBLITERATUS language shards → 00001–00003-of-00005
2. Raw-byte extraction of vision_tower.* from JANG/model-00001-of-00004.safetensors (the only shard containing vision in the 4-shard JANG layout) → split into 00004 (145 tensors) + 00005 (256 tensors) via header parsing without dtype decoding
3. config.json fusion: inject vision_config + capabilities.has_vision from JANG, merge quantization to 87 entries
4. weight_map fusion: 1847 + 501 = 2348, verified 0 duplicate keys, language_model_only: false
5. Copy preprocessor_config.json / video_preprocessor_config.json from JANG; tokenizer family from OBLITERATUS
6. E2E mlx_vlm.load(".") → VisionModel OK (no MTP speculative 31 tensors — excluded for simplicity)

---

🚀 Quickstart

Requirements

pip install -U mlx-vlm   # >=0.6.10
pip install "transformers>=5.14" pillow
# Apple Silicon only — MLX

mlx-vlm (recommended)

from mlx_vlm import load, generate
from PIL import Image

# 1. Load — trust_remote_code required for Qwen3_5
model, processor = load(
    "MrMofer/Qwen3-27B-OBLITERATED-JANG-VISION-MLX-4bit",
    trust_remote_code=True
)
# Local path also works:
# model, processor = load("/path/to/Qwen3-27B-OBLITERATED-JANG-VISION-MLX-4bit", trust_remote_code=True)

# 2a. Text only
messages = [{"role": "user", "content": "Explain photosynthesis in one paragraph."}]
prompt = processor.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, processor, prompt, verbose=False))

# 2b. With image — mlx-vlm auto-inserts <|vision_start|><|image_pad|><|vision_end|>
image = Image.open("foto.jpg").convert("RGB")
messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": image},
        {"type": "text", "text": "Describe this image in one word."}
    ]
}]
prompt = processor.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, processor, prompt, image, verbose=False))

# 2c. With image_url (processor handles PIL internally if you pass path)

Generation tips inherited from OBLITERATUS: temperature 0, repetition_penalty 1.15, max_new_tokens >=2048, enable_thinking false for code tasks yields most deterministic results.

Manual template detail: chat_template.jinja renders <|vision_start|><|image_pad|><|vision_end|> for images and <|video_pad|> for video. Do not inject tokens manually when using processor.apply_chat_template.

OMLX (OpenAI-compatible local server)

Restart OMLX after placing model in ~/.omlx/models/ — it auto-discovers pipeline_tag: image-text-to-text and exposes as Qwen3-27B-OBLITERATED-JANG-VISION-MLX-4bit.

# Text
curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen3-27B-OBLITERATED-JANG-VISION-MLX-4bit",
    "messages": [{"role":"user","content":"Hello, who are you?"}],
    "temperature": 0,
    "max_tokens": 512
  }' | jq -r .choices[0].message.content

# Vision — image_url (local file as base64 or remote URL)
curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen3-27B-OBLITERATED-JANG-VISION-MLX-4bit",
    "messages": [{
      "role": "user",
      "content": [
        {"type":"text","text":"What is in this image?"},
        {"type":"image_url","image_url":{"url":"https://example.com/cat.jpg"}}
      ]
    }],
    "temperature": 0
  }' | jq

OMLX automatically handles preprocessor_config.json / video_preprocessor_config.json preprocessing (longest_edge up to 16777216, shortest 65536).

---

📊 Performance

Measured on Apple Silicon via MLX; unified memory includes model + KV cache for 2k context. Longer context (up to 262k) scales KV linearly.

Quality benchmarks (inherited, not re-measured in hybrid):

• MMLU (text): 81.4% (OBLITERATUS language core unchanged)
• Refusal rate: 0% on standard refusal probes (SVD-ablated)
• Vision hallucination: +1–2% vs JANG native estimated (merger not re-trained for OBLITERATUS language — see Limitations)

---

⚠️ Limitations

• Merger not re-trained — The model.merger projection (10 tensors) is from JANG, not fine-tuned for the OBLITERATUS language distribution. May cause +1–2% vision hallucination vs native JANG on ambiguous images. Text MMLU unaffected.
• deepstack_visual_indexes: [] — Qwen3.5 default, not deep-stacked. Sufficient for single-image VQA/captioning.
• No MTP speculative decoding — 31 tensors (model.mtp.*) from JANG excluded for simplicity. Enable by re-adding if you need speculative streaming.
• Requires mlx-vlm >=0.6.10 and transformers >=5.14 — qwen3_5 support is recent.
• Apple Silicon only — MLX does not run on CUDA/Linux.
• 4-bit quantization loss — negligible for chat/VQA; avoid for tasks requiring exact numeric reproduction.

---

📄 License

Apache 2.0 — same as Qwen/Qwen3-27B and upstream OBLITERATUS/JANG.

Copyright 2025 Alibaba Cloud (Qwen) and contributors
Licensed under the Apache License, Version 2.0
https://www.apache.org/licenses/LICENSE-2.0

See LICENSE for full text. Commercial use permitted; include license and attribution.

---

🙏 Acknowledgements

• Language: OBLITERATUS/Qwen3.8-27B-OBLITERATED by Pliny the Prompter — 6 rounds, 5 SVD directions, residue mining, aggressive uncensor.
• Vision: dealignai/Qwen3.8-27B-JANG_4D-CRACK — full ViT extraction and Qwen3VLProcessor configs.
• Base: Qwen/Qwen3-27B & Qwen/Qwen3-VL by Alibaba Cloud — 64L hybrid attention + 27L ViT architecture.
• Quantization: ml-explore/mlx + mlx-vlm — affine 4-bit packing.
• Built with OMLX local harness (~/.omlx/models/) — E2E verified October 2025.

---

🔗 Repository

• HF: MrMofer/Qwen3-27B-OBLITERATED-JANG-VISION-MLX-4bit
• Local origins:

  ~/.omlx/models/OBLITERATUS/Qwen3.8-27B-OBLITERATED            (14 GB)
  ~/.omlx/models/dealignai/Qwen3.8-27B-JANG_4D-CRACK            (17 GB)
  ~/.omlx/models/OBLITERATUS/Qwen3.8-27B-OBLITERATED-JANG-VISION (15 GB) ← hybrid source
  

If you find this hybrid useful, please star and cite upstream Qwen, OBLITERATUS and JANG.