Step-3.7-Flash-MXFP4-mlx

lemuralabs/Step-3.7-Flash-MXFP4-mlx is an Apple-Silicon MLX MXFP4 tensor-format quantization of stepfun-ai/Step-3.7-Flash.

No fine-tuning, distillation, or retraining was applied. The upstream StepFun checkpoint was downloaded and verified locally, then eligible text and vision .weight tensors were converted with MLX mode="mxfp4" quantization. Tokenizer, chat template, custom Step3.7 Python modules, and non-quantized control tensors are preserved from the source release.

Public Lemura Labs Step-3.7-Flash Variants

Compatibility Status

This upload is a standard MLX MXFP4 safetensors bundle, but it is not yet a drop-in mlx_lm.load(...) or mlx_vlm.load(...) model.

At conversion time, vanilla mlx-lm 0.31.3 and mlx-vlm 0.5.0 did not register model_type: step3p7. Upstream mlx-lm had Step3p5 support, but no Step3p7 model class was found in the installed packages, current upstream trees, open PR search, Hugging Face community page, Reddit LocalLLaMA discussion, or X/Twitter search.

This repository is intended for MLX runtime authors, loader implementers, and researchers who want a verified MXFP4 Step-3.7-Flash tensor bundle. Native inference will require Step3p7 model-class support in MLX/MLX-LM/MLX-VLM or a compatible custom loader.

Mac Inference Guidance

Best current use of this repository on Apple Silicon is:

In short: this is the right artifact for MLX implementers and high-memory Apple-Silicon experiments. For a normal point-and-click local chat app today, users should wait for Step3p7 support in MLX tooling or use a GGUF build made specifically for llama.cpp-compatible runtimes.

Useful upstream/runtime references:

• MLX: https://github.com/ml-explore/mlx
• MLX-LM: https://github.com/ml-explore/mlx-lm
• MLX-VLM: https://github.com/Blaizzy/mlx-vlm
• StepFun base model: https://huggingface.co/stepfun-ai/Step-3.7-Flash

Mac Memory And Storage Recommendations

This MXFP4 bundle is about 101 GB on disk. Runtime memory is not just file size: the loader, KV cache, vision inputs, tokenizer/processor state, temporary tensors, OS memory pressure, and any dequantization or unsupported fallback path can add substantial overhead.

Storage recommendation: keep at least 130 GB free for the repository itself, and more if using Hugging Face cache, duplicate downloads, conversion workspaces, or multiple quant variants. For development, 220 GB or more free space is more comfortable.

Context recommendation: Step-3.7-Flash advertises a very long context window, but full-context local inference will be dominated by KV cache memory. Start with short contexts, then raise context length only after measuring memory use on the target Mac. If the runtime exposes KV-cache limits, set them deliberately.

Model Details

Quantization Recipe

MXFP4 tensors do not use affine biases. Quantized tensors use:

• .weight
• .scales

Conversion summary:

The effective BPW estimate is 4 + 8/32: 4 data bits plus one 8-bit scale per group of 32 values.

Files

• model-00001.safetensors to model-00024.safetensors: text/model shards in MLX MXFP4 tensor format.
• model-vit-00001.safetensors and model-vit-00002.safetensors: vision encoder shards in MLX MXFP4 tensor format.
• model.safetensors.index.json: rewritten safetensors index for the quantized weight/scale tensors.
• mlx_quantization_manifest.json: conversion manifest with quantized/passthrough tensor counts and tensor-level metadata.
• config.json: upstream config with added MLX quantization metadata.
• configuration_step3p7.py, modeling_step3p7.py, processing_step3.py, vision_encoder.py: upstream custom Step3.7 code.
• tokenizer.json, tokenizer_config.json, special_tokens_map.json, chat_template.jinja: upstream tokenizer and prompt assets.

All 26 safetensors shards are required. The index references every model-*.safetensors shard and both model-vit-*.safetensors shards; none are extra convenience files. A downloader needs the full shard set plus model.safetensors.index.json, config.json, tokenizer files, chat template, and custom Step3.7 Python modules for a complete local model bundle.

mlx_quantization_manifest.json is not required for inference, but it is intentionally included for auditability, debugging, and reproducibility of the MXFP4 conversion.

Tensor Inspection

Until Step3p7 support lands in an MLX runtime, use MLX tensor loading for inspection or custom loader development:

import mlx.core as mx

tensors = mx.load("model-00001.safetensors")
prefix = "model.layers.0.self_attn.q_proj"

print(tensors[prefix + ".weight"].shape, tensors[prefix + ".weight"].dtype)
print(tensors[prefix + ".scales"].shape, tensors[prefix + ".scales"].dtype)
print(prefix + ".biases" in tensors)

Representative local verification for that tensor returned:

Limitations

• This is a tensor-format MLX MXFP4 conversion, not a complete native Step3p7 MLX inference implementation.
• Current vanilla mlx-lm and mlx-vlm releases need Step3p7 architecture support before this can be used as a normal one-line load/generate model.
• MXFP4 is more compact than the 8-bit affine variant, but quality has not been benchmarked after conversion.
• Multimodal prompt plumbing depends on future Step3p7 loader/runtime support.
• Behavior, benchmark scores, and deployment claims come from the upstream StepFun release; this quantization has not been re-benchmarked.

Credits

Thank you to both sides of this release:

License: Apache-2.0, following the upstream StepFun release.