Qwen3.5-2B, 4-bit MLX, text only

mlx-community/Qwen3.5-2B-4bit without its vision tower.

Removed: the 297 vision_tower.* tensors (0.66 GB, bf16).
Kept: the other 694 tensors, byte for byte. The config, tokenizer and chat template are unchanged.
Size: 1.06 GB, down from 1.72 GB.
License: Apache-2.0, the license of the source.

Text loaders do not read the vision weights, so text output does not change. Image input does not work with this build.

Why

Made for Pretype, a macOS autocomplete app. This model runs the fix and reply chords on 8 GB Macs. Pretype only sends text, so the vision tower was a download that it never loaded.

Measurements

Pretype's evaluation harness, 2026-10-16. Greedy decoding. Same binary, same rows.

	mlx-community/Qwen3.5-2B-4bit	this build
Fix: identical output, 680 rows	—	680 / 680
Reply: identical output, 570 rows	—	570 / 570
Fix: typo line restored exactly	33%	33%
Download	1.72 GB	1.06 GB

mlx-lm 0.31.3 also gives the same output as the source build.

Use

from mlx_lm import load, generate
model, tokenizer = load("nikimori/Qwen3.5-2B-4bit-slim")
messages = [{"role": "user", "content": "Fix the typos: I recieved teh package yesterday."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=40))

Loading…