Mistral Large 4 (NVFP4)
Run with https://llama.app
llama serve -hf ggml-org/Mistral-Large-4-1T-A52B-NVFP4-GGUF:NVFP4
Source models
Related models
- ggml-org/Mistral-Large-4-1T-A52B-GGUF: GGUFs of the FP8 release, from BF16 down to IQ2_XXS
- mistralai/Mistral-Large-4-1T-A52B: Mistral's FP8 release
Files
| File | Size | Notes |
|---|---|---|
| NVFP4 | 619 GB | Routed experts in the original NVFP4, everything else in BF16 |
| dspark (BF16) | 4.2 GB | DSpark speculative decoding draft head |
| mmproj (BF16, F16) | 3.4 / 3.4 GB | Vision encoder |
Notes
- Mistral Large 4 is a 1T-parameter mixture-of-experts model (128 routed experts, 4 active per token, plus 1 shared expert; ~52B active parameters) with image input and a 1M-token context window.
- The routed experts are copied from Mistral's NVFP4 release without re-quantization. The FP8 layers of the release (attention, indexer query projection, shared experts and the dense first layer) are stored losslessly in BF16, as are the layers that were already BF16.
- The DSpark draft head is 3 MLA layers that propose 3 tokens at a time from the target model's hidden states, with a 2,048-token sliding window.
- For K-quants (Q8_0 down to IQ2_XXS) made from the FP8 release, see ggml-org/Mistral-Large-4-1T-A52B-GGUF.
- Reasoning depth is set with
reasoning_effort:low,mediumorhigh(the default). Reasoning cannot be turned off. - Recommended sampling (stored in the GGUF metadata):
temperature 1.0,top_p 0.95.