prism-coder-9b-mlx-8bit

dcostenco/prism-coder-9b converted to MLX with 8-bit quantization, for Apple Silicon. Text only: the base model's vision tower is not included.

Provenance

• Source: dcostenco/prism-coder-9b at revision 926f129a0ba1818bd69f8d32c2917c84609ab665.
• Quantization: 8 bits, group size 64 (8.501 bits per weight with scales).
• Converted with mlx-lm 0.31.4.dev132+g94cdcae13 and mlx 0.32.2, the runtime bundled with oMLX 0.7.0.
• Converting the same source files again with these versions gave byte-identical output.

To reproduce:

hf download dcostenco/prism-coder-9b --revision 926f129a0ba1818bd69f8d32c2917c84609ab665 \
  --include "*.safetensors" --include "*.json" --include "chat_template.jinja" \
  --local-dir prism-coder-9b-src
python -m mlx_lm convert --hf-path prism-coder-9b-src --mlx-path prism-coder-9b-mlx-8bit \
  -q --q-bits 8 --q-group-size 64

Use with mlx-lm

python -m mlx_lm generate --model dcostenco/prism-coder-9b-mlx-8bit \
  --prompt "Write a Python one-liner that reverses a string." --max-tokens 60

Tested with the versions above: about 59 tokens per second, 9.7 GB peak memory.

Install for oMLX

oMLX serves the models in its model directory (~/.omlx/models by default) under their folder names:

hf download dcostenco/prism-coder-9b-mlx-8bit --local-dir ~/.omlx/models/prism-coder-9b-mlx-8bit

Then restart oMLX.

License

Apache-2.0, the same as the base model.