Sign up

coder3101/Ministral-3-14B-Reasoning-2512-heretic-GGUF

🤗 Hugging Face sourceapache-2.0185 GBGGUF✓ HF checksums availableupdated today
No torrent yet →

Ministral-3-14B-Reasoning-2512-heretic-GGUF

GGUF quantizations of coder3101/Ministral-3-14B-Reasoning-2512-heretic for use with llama.cpp and compatible tools.

Model Description

This is a fine-tuned version of Mistral's Ministral-3-14B-Reasoning-2512 vision-language model. It supports:

  • Text generation with reasoning capabilities (uses [THINK] tokens)
  • Vision/Image understanding (requires the mmproj file)
  • Tool/Function calling

Available Quantizations

Quantization Size Description
BF16 26 GB Full precision (bfloat16)
Q8_0 14 GB 8-bit quantization
Q5_K_M 9.0 GB 5-bit K-quant (medium)
Q4_K_M 7.7 GB 4-bit K-quant (medium) - Recommended

Vision Support

For vision/image understanding, you need to download the mmproj (multimodal projector) file:

  • Ministral-3-14B-Reasoning-2512-heretic-mmproj-bf16.gguf (847 MB)

Chat Template

The model includes a custom chat template with reasoning support. The format uses:

  • [SYSTEM_PROMPT]...[/SYSTEM_PROMPT] - System message
  • [INST]...[/INST] - User messages
  • [THINK]...[/THINK] - Model's reasoning/thinking process
  • [IMG] - Image placeholder for vision inputs
  • [TOOL_CALLS] and [TOOL_RESULTS] - For function calling

Example conversation:

[SYSTEM_PROMPT]You are a helpful assistant.[/SYSTEM_PROMPT][INST]What is 2+2?[/INST][THINK]The user is asking for a simple arithmetic calculation. 2+2=4.[/THINK]The answer is 4.

Usage

Text-only (CLI)

llama-cli -m Ministral-3-14B-Reasoning-2512-heretic-Q4_K_M.gguf \
  -p "[INST]What is the capital of France?[/INST]" \
  -n 256

With Vision Support

llama-mtmd-cli \
  -m Ministral-3-14B-Reasoning-2512-heretic-Q4_K_M.gguf \
  --mmproj Ministral-3-14B-Reasoning-2512-heretic-mmproj-bf16.gguf \
  -p "Describe this image in detail." \
  --image /path/to/image.jpg

With llama-server (OpenAI-compatible API)

llama-server \
  -m Ministral-3-14B-Reasoning-2512-heretic-Q4_K_M.gguf \
  --mmproj Ministral-3-14B-Reasoning-2512-heretic-mmproj-bf16.gguf \
  --port 8080

Then query the API:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "ministral", "messages": [{"role": "user", "content": "What is 2+2?"}]}'

Original Model

This GGUF is based on:

License

Apache 2.0