Flint-MLX-Serve-mixed-4-8bit

An Apple-Silicon MLX pack of Recursive Flint, a 39.8B-total / 2.5B-active coding-agent MoE.

Pack size: ~31 GB. Recommended RAM: 40 GB+ unified memory. Requires mlx-serve with Flint support (in progress).

Recipe

Routed expert banks are affine 4-bit at group size 64. Attention, routers, shared-expert weights, embeddings, and LM head are affine 8-bit at group size 64; norms, vectors, and matrices with fewer than 32 rows remain BF16.

The checkpoint was converted shard-by-shard on CPU from RecursiveSI/flint, retaining the upstream Apache-2.0 license.

mlx-serve quickstart

mlx-serve --model pgmharikrishnan/Flint-MLX-Serve-mixed-4-8bit

OpenAI-compatible request:

curl http://127.0.0.1:11234/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"pgmharikrishnan/Flint-MLX-Serve-mixed-4-8bit","messages":[{"role":"user","content":"Write a Python hello world."}]}'

Results

Teacher-forced fidelity vs BF16 (256 tokens of prose): top-1 agreement 87.1% (pass threshold ≥85%), mean full-vocabulary KL 0.097 nats. ✅ Passes. For comparison, the uniform 4-bit pack scores 61.3% / 0.73 nats, so this mixed pack is the recommended Flint quant.

mlx-serve standard benchmark: pending GPU-locked run.