Sign up

wang-yang/Ornith-1.5-9B-MTPLX-4bit

🤗 Hugging Face sourcetext-generationmit9B params18 GBsafetensors✓ HF checksums availableupdated today
No torrent yet →

Ornith-1.5-9B-MTPLX

A 4-bit MLX/MTPLX quantization of ornith-ai/Ornith-1.5-9B for Apple Silicon. The native one-layer MTP head is included as a quantized sidecar, along with the vision tower.

Quantization

  • Main model: 4-bit affine, group size 64
  • MTP sidecar: 4-bit affine, group size 64 (mtp.safetensors)
  • MTP normalization tensors: BF16
  • Vision tower: BF16 (model-vision.safetensors)

Usage

mtplx start cli --model <path-to-this-model>
mtplx tune --model <path-to-this-model> --retune

The upstream model is licensed MIT. MTPLX: https://github.com/youssofal/MTPLX