SparkMuse-4B MLX 4bit
MLX 4-bit quantization of hcnote/SparkMuse-4B, a creative-writing / novel-generation / roleplay fine-tune of iFlytek Spark X2.5 (spark2_5 architecture, 1M context).
Quantization
- affine, 4-bit, group_size 64 (all 2D tensors including embedding)
- weights: 2.3 GB (BF16 source: 7.7 GB), resident ~2.5 GB
- speed on M2 Ultra (192 GB): ~129 tok/s decode (vs 60 tok/s for the BF16 original)
tie_word_embeddingsis handled by the engine (lm_head shares the quantized embedding)
Usage
Requires mlx-serve 26.9.5+ (native spark2_5 support). mlx-lm 0.31.x does not ship this architecture yet.
mlx-serve pull yachen4ever/SparkMuse-4B-MLX-4bit
mlx-serve serve yachen4ever/SparkMuse-4B-MLX-4bit --port 11234
OpenAI-compatible endpoint at http://127.0.0.1:11234/v1/chat/completions.
Notes
- The upstream repo is tagged
nsfw/ sensitive content by the author; the base model card describes a creative-writing oriented fine-tune. Use at your own discretion. - The upstream
modeling_spark.py/configuration_spark.py(transformers custom code) are not needed: the engine implements the architecture natively.