Mistral-Small-4-119B-2603-heretic GGUf Quantizations (imatrix)

Model Information

Key Features

• MoE Architecture: 128 experts with 4 active per token (6.5B activated parameters)
• Multimodal: Accepts both text and image input, produces text output
• Reasoning Mode: Toggle between fast instant reply and reasoning mode with test-time compute
• Function Calling: Native function calling with JSON output
• Agentic: Best-in-class agentic capabilities
• System Prompts: Strong adherence and support for system prompts
• Speed-Optimized: Best-in-class performance and speed
• Large Context Window: 256k context length supported
• Apache 2.0 License: Open-source for commercial and non-commercial use
• Multilingual: Supports English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, Arabic and more

Heretic Model Statistics

imatrix Quantizations

These models were quantized using an importance matrix (imatrix) computed from a reference corpus. The imatrix helps preserve quality for important token patterns, typically yielding better results at low bitrates compared to standard quantization.

The imatrix file was generated using the Q8_0 quantization of the heretic model on the following calibration dataset:

Calibration Dataset: bartowski1182/calibration_datav5.txt Reference imatrix: Mistral-Small-4-119B-2603-heretic-imatrix.gguf (118 MB)

Available Quantizations