Run it on your Mac
curl -fsSL https://rapidmlx.com/install.sh | bash
rapid-mlx serve g9v3-39a5b-4bit
curl -s http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"default","messages":[{"role":"user","content":"Say hello"}]}'
Use it from OpenCode / Claude Code / Codex: rapid-mlx agents <name> --setup · Agent setup
Download the Desktop app · 32 GB unified memory catalog minimum; the weights alone are 22.8 GB.
G9v3-39A5B — MLX 4-bit
MLX conversion of ai9stars/G9v3-39A5B, a 39B Mixture-of-Experts causal LM with 5B parameters active per token (32 of 320 routed experts + 1 shared expert), gated GQA attention and a 128K context.
The g9v3 architecture is not in mlx-lm; it is served by the vendored MLX
backbone in rapid-mlx
(vllm_mlx/models/g9v3.py), which registers itself into mlx-lm's model
lookup. Plain mlx_lm.load without rapid-mlx will not recognise the
model_type.
Quantization recipe (mlx_lm.convert(..., quantize=True, q_bits=4, q_group_size=64),
mlx-lm 0.31.3, with the vendored module registered): the 320 routed experts
per layer are 4-bit / group 64 (affine); attention, the dense first-layer
MLP, the shared experts, embeddings and lm_head are 8-bit / group 64 —
4.681 bits per weight overall. Uniform 4-bit was measured to hurt this
architecture's attention projections badly (mean top-1 agreement with the
bf16 reference over all prompt positions 0.80 → 0.90 with this recipe;
8-bit everywhere 0.96, bf16 MLX itself 0.955). The MLX
port was checked against the upstream trust_remote_code implementation
on identical random weights (max |Δlogit| < 1e-5 in fp32) and on the
released weights (see the rapid-mlx PR for the bf16 agreement tables).
Usage
curl -fsSL https://rapidmlx.com/install.sh | bash
rapid-mlx serve g9v3-39a5b-4bit
Then talk to it through the OpenAI-compatible API on http://localhost:8000.
Reasoning uses Qwen3-style <think> blocks (enable_thinking toggles it);
tool calls use the model's native XML format, parsed server-side.
License
Apache-2.0, following the upstream release.