Ministral-3-14B-Reasoning-2512 · GGUF Q4_K_M

Quantized, converted, and evaluated by PBH Applied Systems, LLC — Applied AI/ML Consulting · LLM Optimization & Deployment · Quantized AI Infrastructure

🔬 This repository is part of a production-oriented evaluation series. Every model published under pbhappliedsystems has been independently evaluated using quant_eval v7.21 — a proprietary behavioral evaluation harness developed by PBH Applied Systems. Scores measure real agent-adjacent task performance across structured output, tool dispatch, multi-turn state retention, and multi-step planning families — not perplexity or benchmark leaderboard proxies.

🧠 This is a reasoning-architecture model. Ministral-3-14B-Reasoning-2512 generates internal chain-of-thought traces before producing its final answer. This has significant implications for inference speed, tool-calling behavior, and quantization tradeoffs — all documented in detail below.

---

Try This Model in the Live AI Agent Demo

Launch the PBH Applied Systems AI Agent Demo →

This model is part of the PBH Applied Systems live AI Agent Demo, where visitors can test evaluated quantized open-weight models across production-style agent workflows: reasoning and analysis, document intelligence, and code automation.

The demo uses quant_eval results to show how model selection changes by task. A model that performs well for long-context document analysis may not be the best choice for hard multi-step planning, strict tool-use workflows, or production code generation. Each deployed model is evaluated for practical agent behavior, including coherence, instruction following, reasoning, task completion, structured output reliability, tool-use behavior, and quantization impact.

For this repository, the Q4_K_M variant represents the deployment-focused model: smaller, faster, and more cost-efficient than the F16 baseline. The evaluation results below explain where this quantized model preserves useful behavior, where quantization introduces risk, and what guardrails are recommended before production deployment.

The purpose of the demo is simple: let prospects test the same kind of evaluated quantized models that PBH Applied Systems deploys for real agentic AI systems.

---

Model Description

This repository contains the 4-bit quantized (Q4_K_M) GGUF of mistralai/Ministral-3-14B-Reasoning-2512, a 14-billion parameter reasoning-tuned model from Mistral AI (December 2025 release). This is the reasoning variant of the Ministral-3-14B architecture — distinct from the instruction-tuned variant (pbhappliedsystems/ministral-3-14b-instruct-2512-gguf-Q4-K-M) in that it performs extended internal deliberation before generating output.

The Q4_K_M format applies 4-bit quantization with K-quant medium precision. As documented in the evaluation section below, Q4_K_M quantization has a pronounced and measurable effect on this model's reasoning behavior — compressing the chain-of-thought dramatically and altering failure modes in ways that differ substantially from the F16 baseline.

The full-precision F16 baseline is published separately at pbhappliedsystems/ministral-3-14b-reasoning-2512-gguf-F16.

Key Characteristics

• Parameters: 14B
• Architecture: Reasoning (extended chain-of-thought)
• Format: GGUF Q4_K_M
• File size: 8.24 GB
• SHA256: e7171d96748ddc948fd6d9edb3d1c6e3f9ba6b855ff964aee98519788da330c2
• Minimum VRAM (GPU inference): ~10–12 GB (T4 class or better)
• Recommended GPU tier: NVIDIA T4 (16 GB) · RTX 3080/4080 · A10G
• Context window: 32,768 tokens (per base model specification)
• Q4_K_M avg inference time (eval hardware): 1.18 sec/case on RTX 4090
• F16 avg inference time (eval hardware): 65.67 sec/case on RTX 4090

---

⚡ The Quantization Speed Story — 55.7× Faster, With Tradeoffs

The single most striking finding from this evaluation is the inference timing differential between precision levels:

The chain-of-thought is the cost at F16. The F16 reasoning model spends 35–176 seconds per case generating internal deliberation traces before producing output. Fuzz case fuzz_0000 took 176.47 seconds at full precision. The json family averaged over 105 seconds per case.

Q4_K_M compresses the reasoning dramatically. At 1.18 sec/case average, the Q4_K_M variant is not just faster — it is the fastest model in the PBH Applied Systems evaluated series, faster even than Qwen2.5-3B-Instruct. This speed comes from the effective compression or elimination of the extended chain-of-thought generation. Whether this constitutes a deployment advantage or a capability regression depends entirely on your use case, and the evaluation data below provides the evidence to make that determination.

---

PBH Applied Systems Evaluation — quant_eval v7.21

Evaluation conducted by PBH Applied Systems, LLC using quant_eval v7.21 Run ID: 20260209_233252 · Fixtures: golden_oracle_fixtures_v7_21 (SHA256: 6d71a0b9147c...) · Seed: 42 Hardware: NVIDIA RTX 4090 · Total rows evaluated: 84 (42 F16 · 42 Q4_K_M)

Aggregate Scores (Q4_K_M)

Scores are normalized to [0.0 – 1.0]. Higher is better.

Note on avg_secs: The 33.43 sec figure that appears in cross-model comparison tables is the arithmetic mean of both runners (F16: 65.67 + Q4_K_M: 1.18 ÷ 2). The Q4_K_M standalone figure of 1.18 sec/case is the operationally relevant number for deployment planning.

Per-Family Pass Rates

F16 Baseline (full_weight_transformers)

Q4_K_M (quantized_llama_cpp)

---

Key Findings — Reasoning Architecture Under Quantization

Finding 1: json_multistep — Hard Case Capability Regression

The aggregate pass rate (0.600) is identical across both runners, but the case-level breakdown reveals a meaningful capability difference:

The F16 passes the hard case; the Q4_K_M fails it. The F16 model's extended chain-of-thought reasoning (110.73 sec on ms_hard_01) enables it to work through the harder planning problem correctly. The Q4_K_M variant, with its compressed reasoning at 3.39 sec, loses this capability precisely where it matters most. The easy case trade also inverts — Q4_K_M recovers ms_easy_01 while F16 misses it, suggesting a qualitative shift in how planning is approached rather than uniform degradation.

Practical implication: For multi-step planning tasks at medium-to-hard difficulty, the F16 is meaningfully more capable. If planning correctness on harder cases is a deployment requirement, the F16 or an external validation loop is necessary.

Finding 2: toolcall_only — Architectural Incompatibility

toolcall_only fails on both runners (0.000), but the failure signatures differ in an instructive way:

This is an architectural finding, not a quantization finding. The reasoning model's chain-of-thought output is structurally incompatible with bare schema-only tool dispatch. At F16, the extended reasoning trace interferes with tool name extraction on one of two cases (0.5). At Q4_K_M, tool name recognition is actually perfect (1.0) — the compressed reasoning improves extraction — but argument construction remains broken at both precision levels.

The root cause is the thinking trace: a reasoning model generating chain-of-thought tokens cannot cleanly emit a bare {"tool_name": "...", "args": {...}} JSON block without prose scaffolding. This is not a bug in the quantization; it is a characteristic of reasoning architecture that toolcall (1.000 at both runners) works correctly — when tool dispatch is embedded within a broader response, the reasoning traces are absorbed into the response structure and the tool call itself is valid.

Practical implication: Do not use this model for bare tool-call dispatch at any precision level without a schema enforcement layer. Use toolcall mode (tool call embedded in a broader response) for reliable tool use.

Finding 3: MCQ — Chain-of-Thought Wrapping vs. Answer Accuracy

Both runners fail mcq_02, but for entirely different reasons that reveal the reasoning architecture's behavior:

The F16 model wraps its MCQ response in a chain-of-thought + code fence block, causing the choice extractor to capture the fence delimiter instead of the answer letter. The Q4_K_M model suppresses this wrapping (consistent with its compressed reasoning behavior) but in doing so, loses whatever deliberation led to the correct answer at F16 — and selects the wrong choice.

Both fail mcq_02; only one of them had a correct answer buried inside an extraction-unfriendly format.

---

Signal-Level Diagnostics

Q4_K_M — json_multistep

Note on checks_consistent_ok: F16 achieves 1.000 on this signal; Q4_K_M drops to 0.800. This is the quantization-driven consistency regression on ms_hard_01.

Q4_K_M — stateful_followup

Q4_K_M — toolcall_only

Q4_K_M — mixed_brief_json

---

Recommended Use Cases

✅ Deploy with Confidence (Q4_K_M)

• Stateful multi-turn agents — Perfect two-turn state retention (1.000) at 0.46 sec/case. Reliable and fast.
• Structured JSON outputs (single-step) — bucket_score avg of 10.000 on both json and fuzz; valid structured outputs at 1.50 sec avg.
• Hybrid brief + JSON responses — mixed_brief_json passes at 1.000 in 0.42 sec. The fastest mixed-output result in the evaluated series.
• Tool-calling with response scaffolding — toolcall passes at 1.000. Embed tool dispatch in a structured response rather than bare JSON output.
• JSON multi-step at easy-to-medium difficulty with validation loop — ms_easy (partial) and ms_med cases pass reliably; use with external validation for production.
• High-throughput structured inference — At 1.18 sec/case average, Q4_K_M is exceptionally fast for a 14B model. Suitable for batch processing pipelines where throughput matters.

⚠️ Use with Guardrails (Q4_K_M)

• Hard multi-step planning — ms_hard_01 fails at Q4_K_M (passes at F16). Use with an external validator or upgrade to F16 for harder planning horizons.
• MCQ and single-choice extraction — bucket_score 0.800 on both runners. Add a post-processing extraction layer to handle potential reasoning token wrapping.

❌ Not Recommended (Q4_K_M or F16)

• Bare tool-call dispatch (schema-only output) — toolcall_only fails at 0.000 on both runners. This is an architectural limitation of reasoning models, not a quantization artifact. Use toolcall mode instead.
• Applications requiring full chain-of-thought traces — Q4_K_M's compressed reasoning suppresses the deliberation that makes this model variant valuable for hard reasoning tasks. Use F16 if the thinking trace itself is part of the deliverable.

---

F16 vs. Q4_K_M — When to Choose Each

---

Hardware Requirements

---

Usage

Installation

pip install llama-cpp-python huggingface_hub

For GPU acceleration (CUDA):

CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python --force-reinstall --no-cache-dir

Python — llama-cpp-python

from huggingface_hub import hf_hub_download
from llama_cpp import Llama

model_path = hf_hub_download(
    repo_id="pbhappliedsystems/ministral-3-14b-reasoning-2512-gguf-Q4-K-M",
    filename="ministral-3-14b-reasoning-2512-gguf-Q4-K-M.gguf"
)

llm = Llama(
    model_path=model_path,
    n_ctx=8192,       # Increase up to 32768; reasoning traces consume context
    n_gpu_layers=-1,  # -1 offloads all layers to GPU
    verbose=False,
)

response = llm.create_chat_completion(
    messages=[
        {
            "role": "system",
            "content": "You are a precise reasoning assistant. Think through the problem carefully before responding."
        },
        {
            "role": "user",
            "content": "Analyze the following clause for logical inconsistencies and return a structured JSON summary: ..."
        }
    ],
    temperature=0.7,
    max_tokens=2048,  # Reasoning models may emit longer outputs even at Q4_K_M
)

print(response["choices"][0]["message"]["content"])

For tool-calling, always use response-scaffolded mode (see evaluation findings — bare tool dispatch is unreliable):

import json, re
from huggingface_hub import hf_hub_download
from llama_cpp import Llama

model_path = hf_hub_download(
    repo_id="pbhappliedsystems/ministral-3-14b-reasoning-2512-gguf-Q4-K-M",
    filename="ministral-3-14b-reasoning-2512-gguf-Q4-K-M.gguf"
)

llm = Llama(model_path=model_path, n_ctx=4096, n_gpu_layers=-1, verbose=False)

def call_tool_scaffolded(prompt: str) -> dict:
    """
    Use response-scaffolded tool calling for reasoning models.
    toolcall pass rate = 1.000; toolcall_only pass rate = 0.000.
    Embed the tool call instruction in a broader response prompt.
    """
    response = llm.create_chat_completion(
        messages=[
            {
                "role": "system",
                "content": (
                    "You are a tool-calling assistant. When you determine a tool is needed, "
                    "briefly explain your reasoning, then output the tool call as:\n"
                    "TOOL_CALL: {\"tool_name\": \"...\", \"args\": {...}}"
                )
            },
            {"role": "user", "content": prompt}
        ],
        temperature=0.0,
        max_tokens=512,
    )
    raw = response["choices"][0]["message"]["content"]
    # Extract structured tool call from scaffolded response
    match = re.search(r'TOOL_CALL:\s*(\{.*?\})', raw, re.DOTALL)
    if match:
        return json.loads(match.group(1))
    raise ValueError(f"No tool call found in response: {raw}")

result = call_tool_scaffolded("Calculate 5 plus 10 using the add tool.")

CLI — llama-cli

# One-shot reasoning prompt
llama-cli \
  --model ministral-3-14b-reasoning-2512-gguf-Q4-K-M.gguf \
  --chat-template mistral \
  --system-prompt "You are a precise reasoning assistant. Think through the problem carefully." \
  --prompt "Analyze the following data and identify the three most significant risk factors. Return a JSON array." \
  --n-predict 2048 \
  --ctx-size 8192 \
  --n-gpu-layers -1 \
  --temp 0.15

For server deployment (OpenAI-compatible endpoint):

llama-server \
  --model ministral-3-14b-reasoning-2512-gguf-Q4-K-M.gguf \
  --chat-template mistral \
  --ctx-size 8192 \
  --n-gpu-layers -1 \
  --port 8080 \
  --host 0.0.0.0

Query via the OpenAI-compatible API:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-required")

response = client.chat.completions.create(
    model="ministral-3-14b-reasoning-2512-gguf-Q4-K-M",
    messages=[{"role": "user", "content": "Your prompt here"}],
    temperature=0.7,
)
print(response.choices[0].message.content)

---

Artifact Provenance

Both artifacts were produced from mistralai/Ministral-3-14B-Reasoning-2512 using a custom-built llama.cpp conversion and quantization pipeline developed by PBH Applied Systems.

---

Evaluation Methodology

quant_eval v7.21 is a proprietary behavioral evaluation harness developed by PBH Applied Systems. It evaluates both the full-precision (F16) and quantized variants against an identical fixture set, enabling direct comparison of capability retention across quantization levels.

Fixture set: golden_oracle_fixtures_v7_21 (SHA256: 6d71a0b9147c079371b02a94f3c149eb78a6adc03dc16ff6833b964fbf4174f0)

Evaluation hardware: NVIDIA RTX 4090 (24 GB VRAM) Evaluation date: February 9, 2026 quant_eval seed: 42

---

🔬 About quant_eval & This Evaluation Series

quant_eval is a proprietary behavioral evaluation harness developed by PBH Applied Systems, LLC. It measures real agent-adjacent task performance across structured output, tool dispatch, multi-turn state retention, and multi-step planning — not perplexity or leaderboard proxies. Every model published under pbhappliedsystems has been independently evaluated using quant_eval before being recommended for any production role.

See it in action: Live AI Agent Demo → The demo runs production-style agent workflows powered by open-weight models selected through the quant_eval evaluation pipeline.

Need a deployment recommendation? Not sure which quantization level is right for your hardware, latency target, or agent type? → pbhappliedsystems.com

---

Evaluated and published by PBH Applied Systems, LLC · patrick@pbhappliedsystems.com

---

About PBH Applied Systems

PBH Applied Systems, LLC is an Oklahoma City–based applied machine learning and AI systems company specializing in production-grade model evaluation, quantization pipelines, agentic AI infrastructure, and scalable AI-driven application development. The organization operates with a strong emphasis on engineering rigor, reproducibility, and real-world deployment constraints — particularly in environments where performance, cost efficiency, and reliability must be balanced against available hardware and budget.

Founder & Principal AI/ML Systems Architect — Patrick Hill

PBH Applied Systems was founded by Patrick Hill, a Data Scientist and AI/ML Engineer with 10+ years of experience delivering advanced analytics, predictive modeling, and decision-support solutions across high-stakes operational environments. Patrick holds a Master of Science in Software Engineering with concentrations in Artificial Intelligence and Machine Learning and a B.S. in Business Finance.

Technical expertise spans:

• Languages & Data: Python, SQL, Linux, Pandas, NumPy, scikit-learn
• ML & Modeling: Supervised and unsupervised learning, neural networks, NLP, transformers, regression, classification, forecasting, and feature engineering
• AI/ML Frameworks: PyTorch, TensorFlow/Keras, HuggingFace Transformers, GGUF, llama.cpp, BitsAndBytes, PEFT, QLoRA
• Deployment & MLOps: Flask APIs, Docker, CI/CD pipelines, REST endpoints, streaming inference, version control
• Data Platforms: Jupyter, Databricks, Power BI, Matplotlib
• Quantization: GGUF conversion, Q4_K_M / Q5_K_M / Q8_0 strategies, adapter-per-model evaluation architecture

Published Author

Patrick is the author of Applied Machine Learning: Concepts, Tools, and Case Studies — a 1,200+ page practitioner-oriented textbook covering statistical modeling, supervised and unsupervised learning, neural networks, NLP, and real-world decision-support case studies. Adopted as required reading for CSC 373 – Machine Learning at the University of Advancing Technology.

Core Service Areas

1. LLM Optimization & Deployment — End-to-end GGUF conversion and quantization with custom llama.cpp pipelines and adapter-per-model architecture.

2. AI Evaluation Frameworks — Proprietary behavioral evaluation via quant_eval: per-family pass rates, F16 vs. quantized delta analysis, failure cluster diagnostics, deployment recommendations.

3. Agentic AI Infrastructure — LlamaIndex ReAct agents, Flask orchestration, serverless GPU inference, full pipeline from model selection to production serving.

4. Scalable AI Application Development — Multimodal applications (quantized LLMs + Whisper + BLIP), Dockerized Flask APIs, advanced time-series forecasting with custom attention mechanisms, Bayesian hyperparameter optimization, and FinBERT sentiment fusion.

5. ML Pipeline Design & Analytics — Feature engineering, forward-chaining cross-validation, KPI dashboards, analytical governance at scale.

6. Model & Agent Cataloging — Structured catalog publishing with reproducible artifacts and clear performance tradeoff documentation.

---

📞 Work With PBH Applied Systems

The findings documented in this card — the 55.7× inference speedup, the hard planning case regression, the reasoning architecture's incompatibility with bare tool dispatch, and the MCQ failure mode divergence between precision levels — are all signals that only emerge from systematic, production-oriented evaluation. None of them appear in perplexity scores or benchmark leaderboards.

If you are deploying a reasoning model in production, the gap between F16 and Q4_K_M behavior is not uniform. It is task-specific, signal-level, and only visible when you measure it.

👉 Book a Scoping Call — Discuss your reasoning model selection, quantization strategy, or deployment architecture directly with Patrick.

👉 Request an Evaluation Report — A full quant_eval behavioral audit: per-family pass rates, F16 vs. quantized delta analysis, failure cluster diagnostics, and a deployment recommendation. Engagements from $2,500.

Connect

---

License

This GGUF repository inherits the license of the base model: Apache 2.0 — mistralai/Ministral-3-14B-Reasoning-2512

The quant_eval evaluation methodology, fixture set, and scoring framework are proprietary to PBH Applied Systems, LLC and are not included in this repository.

---

GGUF conversion, quantization, and behavioral evaluation performed by PBH Applied Systems, LLC · quant_eval v7.21 · Run ID: 20260209_233252