Mellum2-12B-A2.5B-Thinking-mlx

This is an MLX version of JetBrains/Mellum2-12B-A2.5B-Thinking, the reasoning-augmented Mixture-of-Experts coding assistant from JetBrains. The weights are kept in their native bfloat16 precision, so the model behaves exactly like the original checkpoint.

Mellum 2 uses 64 experts with 8 active per token (about 2.5B active parameters out of 12B), a mix of sliding-window and full-attention layers, and a 131,072-token context window. It emits its reasoning inside <think>...</think> blocks before the final answer.

Requirements

The mellum architecture is not supported by the stock mlx-lm code yet.

Until it is supported upstream, install this fork of mlx-lm from source:

pip install git+https://github.com/jedisct1/mlx-lm

Or run it directly with uv:

uvx --from git+https://github.com/jedisct1/mlx-lm mlx_lm.server

Use with mlx-lm

Quick test:

uvx --from git+https://github.com/jedisct1/mlx-lm \
  mlx_lm.generate --model jedisct1/Mellum2-12B-A2.5B-Thinking-mlx \
  --prompt "How are you doing today?" \
  --max-tokens 16384 \
  --temp 0.6 --top-p 0.95 --top-k 20

Starting the server:

uvx --from git+https://github.com/jedisct1/mlx-lm \
  mlx_lm.server --model jedisct1/Mellum2-12B-A2.5B-Thinking-mlx \
  --max-tokens 16384 \
  --temp 0.6 --top-p 0.95 --top-k 20  

The recommended sampling settings from JetBrains are temperature=0.6, top_p=0.95, top_k=20.

Using this setup with the Swival.dev harness

Install swival.dev:

uv tool install swival

Then:

swival --provider llamacpp --model jedisct1/Mellum2-12B-A2.5B-Thinking-mlx

License

Apache 2.0, inherited from the original model.