Nemotron-Nano-12B-v2-VL-4bit

4-bit affine quantization of nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16, produced with FFAI 0.1.0's ffai convert (mlx-affine format, group_size=64).

Conversion

ffai convert nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-BF16 --bits 4 \
    --upload-repo ekryski/Nemotron-Nano-12B-v2-VL-4bit

See also

• FFAI — fast Apple Silicon LLM inference. Model.load("ekryski/Nemotron-Nano-12B-v2-VL-4bit") runs this checkpoint end-to-end.
• FFAI quickstart
• FFAI quantization docs