Mistral-Small-4-119B-2603-heretic GGUf Quantizations
Model Information
| Field | Value |
|---|---|
| Base Model | Mistral Small 4 119B 2603 by Mistralai |
| Heretic Model | Mistral Small 4 119B 2603 Heretic by darkc0de |
| Architecture | Mistral4 (MoE, MLA attention) |
| Total Parameters | 119B |
| Active Parameters | 6.5B per token |
| Experts | 128 experts, 4 active |
| Context Length | 256k tokens |
Key Features
- MoE Architecture: 128 experts with 4 active per token (6.5B activated parameters)
- Multimodal: Accepts both text and image input, produces text output
- Reasoning Mode: Toggle between fast instant reply and reasoning mode with test-time compute
- Function Calling: Native function calling with JSON output
- Agentic: Best-in-class agentic capabilities
- System Prompts: Strong adherence and support for system prompts
- Speed-Optimized: Best-in-class performance and speed
- Large Context Window: 256k context length supported
- Apache 2.0 License: Open-source for commercial and non-commercial use
- Multilingual: Supports English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, Arabic and more
Heretic Model Statistics
| Metric | Value |
|---|---|
| KL Divergence | 0.0167 (0 for refusable tokens by definition) |
| Refusals | 27/100 (98/100 original model) |
Available Quantizations
Standard quantizations without importance matrix (imatrix) optimization.
| File | Type | Size | BPW | Description |
|---|---|---|---|---|
Mistral-Small-4-119B-2603-heretic-BF16.gguf |
BF16 | 221.6 GiB | 16.00 | Base model, highest quality |
Mistral-Small-4-119B-2603-heretic-Q8_0.gguf |
Q8_0 | 117.8 GiB | 8.00 | Very high quality |
Mistral-Small-4-119B-2603-heretic-Q6_K.gguf |
Q6_K | 91.0 GiB | 6.00 | High quality |
Mistral-Small-4-119B-2603-heretic-Q5_K_M.gguf |
Q5_K_M | 78.5 GiB | 5.37 | Good quality/size balance |
Mistral-Small-4-119B-2603-heretic-Q5_K_S.gguf |
Q5_K_S | 76.3 GiB | 5.25 | |
Mistral-Small-4-119B-2603-heretic-Q4_K_M.gguf |
Q4_K_M | 67.2 GiB | 4.58 | Recommended |
Mistral-Small-4-119B-2603-heretic-Q4_K_S.gguf |
Q4_K_S | 63.1 GiB | 4.37 | |
Mistral-Small-4-119B-2603-heretic-IQ4_NL.gguf |
IQ4_NL | 63.0 GiB | 4.50 | Non-linear quantization |
Mistral-Small-4-119B-2603-heretic-Q3_K_L.gguf |
Q3_K_L | 57.4 GiB | 3.70 | |
Mistral-Small-4-119B-2603-heretic-Q3_K_M.gguf |
Q3_K_M | 53.1 GiB | 3.63 | |
Mistral-Small-4-119B-2603-heretic-Q3_K_S.gguf |
Q3_K_S | 47.8 GiB | 3.41 | |
Mistral-Small-4-119B-2603-heretic-Q2_K.gguf |
Q2_K | 40.4 GiB | 2.96 | Low memory usage |