English注册

spicyneuron/Qwen3.5-122B-A10B-MLX-4.6bit

来源：🤗text-generationapache-2.0122B 参数激活 10B244 GBsafetensors✓ 15 个校验和193 天前更新
或提交种子

Qwen3.5-122B-A10B optimized for MLX. This quant does not support image input.

For vision support: https://huggingface.co/spicyneuron/Qwen3.5-122B-A10B-MLX-4.7bit-vision

EDIT: Updated chat template to enable better prompt caching.

Usage

# Start server at http://localhost:8080/v1/chat/completions
uvx --from mlx-lm mlx_lm.server \
  --host 127.0.0.1 \
  --port 8080 \
  --model spicyneuron/Qwen3.5-122B-A10B-MLX-4.6bit

Methodology

Quantized with a mlx-lm fork, drawing inspiration from Unsloth/AesSedai/ubergarm style mixed-precision GGUFs. MLX quantization options differ than llama.cpp, but the principles are the same:

  • Sensitive layers like MoE routing, attention, and output embeddings get higher precision
  • More tolerant layers like MoE experts get lower precision