Mistral Large 4 · NVFP4

Original Mistral Large 4 NVFP4 weights, hosted by Nativ Community. A multimodal Mixture-of-Experts model for text and image understanding, reasoning, coding, and tool use.

The weights retain Mistral's original precision and packing: NVFP4 experts, FP8 dense layers, and a bundled BF16 DSpark draft head. Source revision: bb6d00fe249b0e876f0cffc6b85014be02de3e68.

Usage

Download the checkpoint:

hf download nativ-community/Mistral-Large-4-1T-A52B-NVFP4 \
  --local-dir ./Mistral-Large-4-1T-A52B-NVFP4

MLX-VLM

Mistral Large 4 support is available in MLX-VLM, including distributed inference across Macs, continuous batching, and bundled DSpark.

pip install -U git+https://github.com/Blaizzy/mlx-vlm.git@main

This checkpoint uses Mistral's native params.json. Before loading with MLX-VLM, prepare a local config.json with model_type: mistral_large4, text_config, and vision_config. See the model instructions for the configuration requirements, standard distributed runner, and DSpark API.

Distributed inference across Macs

Use MLX's JACCL backend over Thunderbolt 5 RDMA. Connect the Macs with Thunderbolt 5, enable RDMA on each device, and configure SSH access following the MLX distributed setup guide.

On every Mac, use matching MLX and MLX-VLM versions and the same absolute paths for the Python environment, model directory, and MLX-VLM checkout. The model directory must contain the normalized config.json described above. Clone the repository to obtain the standard distributed runner:

git clone https://github.com/Blaizzy/mlx-vlm.git /path/to/mlx-vlm
/path/to/venv/bin/python -m pip install -e /path/to/mlx-vlm

From one Mac, generate a hostfile for both devices. Replace the hostnames with your SSH hosts and apply the network setup commands printed by the utility:

mlx.distributed_config \
  --hosts mac-1.local,mac-2.local \
  --over thunderbolt --backend jaccl \
  --output /path/to/hosts.json

Launch inference from that Mac; the launcher starts a worker on each host:

mlx.launch \
  --backend jaccl \
  --hostfile /path/to/hosts.json \
  --cwd /path/to/mlx-vlm \
  --python /path/to/venv/bin/python \
  /path/to/mlx-vlm/examples/sharded_generate.py \
  --backend jaccl \
  --model /path/to/Mistral-Large-4-1T-A52B-NVFP4 \
  --prompt "Write a Python function that returns the first n Fibonacci numbers." \
  --max-tokens 512

For image input, add --image /path/to/image.jpg and place the image at the same path on every Mac. Experts are partitioned across devices; dense and vision layers are replicated. The runner prints prompt speed, decode speed, and peak MLX memory.

Recommended settings

Mistral recommends temperature=1.0 and top_p=0.95. Reasoning effort supports low, medium, and high; the source default is high.

License

Mistral Ecosystem Open-Weight License. See the original model card for architecture, evaluations, and intended uses.