Ministral-3-14B-Instruct-2512 · GGUF Q4\_K\_M

Quantized, converted, and evaluated by PBH Applied Systems, LLC

— Applied AI/ML Consulting · LLM Optimization & Deployment · Quantized AI Infrastructure

🔬 This repository is part of a production-oriented evaluation series. Every model published under pbhappliedsystems has been independently evaluated using quant_eval v7.21 — a proprietary behavioral evaluation harness developed by PBH Applied Systems. Scores measure real agent-adjacent task performance across structured output, tool dispatch, multi-turn state retention, and multi-step planning families — not perplexity or benchmark leaderboard proxies.

Try This Model in the Live AI Agent Demo

Launch the PBH Applied Systems AI Agent Demo →

This model is part of the PBH Applied Systems live AI Agent Demo, where visitors can test evaluated quantized open-weight models across production-style agent workflows: reasoning and analysis, document intelligence, and code automation.

The demo uses quant_eval results to show how model selection changes by task. A model that performs well for long-context document analysis may not be the best choice for hard multi-step planning, strict tool-use workflows, or production code generation. Each deployed model is evaluated for practical agent behavior, including coherence, instruction following, reasoning, task completion, structured output reliability, tool-use behavior, and quantization impact.

For this repository, the Q4_K_M variant represents the deployment-focused model: smaller, faster, and more cost-efficient than the F16 baseline. The evaluation results below explain where this quantized model preserves useful behavior, where quantization introduces risk, and what guardrails are recommended before production deployment.

The purpose of the demo is simple: let prospects test the same kind of evaluated quantized models that PBH Applied Systems deploys for real agentic AI systems.

Model Description

This repository contains the 4-bit quantized (Q4\_K\_M) GGUF of mistralai/Ministral-3-14B-Instruct-2512, a 14-billion parameter instruction-tuned model from Mistral AI (December 2025 release).

The Q4\_K\_M format applies 4-bit quantization with K-quant medium precision, targeting a balance of inference speed and output fidelity suitable for deployment on consumer and professional GPU hardware. The full-precision F16 baseline is published separately at pbhappliedsystems/ministral-3-14b-instruct-2512-gguf-F16.

Key Characteristics

Parameters: 14B
Format: GGUF Q4\_K\_M
File size: 8.24 GB
SHA256: a23910514ee512aa28db8dddd390c26a73b9c318dcdec374ae02d722d9658749
Minimum VRAM (GPU inference): ~10–12 GB (T4 class or better)
Recommended GPU tier: NVIDIA T4 (16 GB) · RTX 3080/4080 · A10G
Context window: 32,768 tokens (per base model specification)
Inference speed (eval hardware): avg 3.77 sec/case on RTX 4090

PBH Applied Systems Evaluation — quant\_eval v7.21

Evaluation conducted by PBH Applied Systems, LLC using quant_eval v7.21

Run ID: 20260209_170235 · Fixtures: golden_oracle_fixtures_v7_21 (SHA256: 6d71a0b9147c...) · Seed: 42

Hardware: NVIDIA RTX 4090 · Total rows evaluated: 84 (42 F16 · 42 Q4\_K\_M)

Aggregate Scores (Q4\_K\_M)

Scores are normalized to [0.0 – 1.0]. Higher is better.

| Dimension | Score |

|---|---:|

| Task Completion | 0.6809 |

| Reasoning | 0.9148 |

| Coherence | 0.9259 |

| Instruction Following | 0.9689 |

| Avg inference time | 3.77 sec/case |

Per-Family Pass Rates

The evaluation runs 8 task families. Pass rate is a conjunction of all gating signals for that family — a strict, production-oriented measure. Families marked n/a use bucket scoring rather than binary pass/fail.

F16 Baseline (`full_weight_transformers`)

|---|---:|---:|---|

| json\_multistep | 5 | 0.600 | Tier-1 gating: schema, checks, stop semantics, oracle equivalence |

| stateful\_followup | 2 | 1.000 | Both turns parse and match expected state |

| toolcall\_only | 2 | 1.000 | Tool name + args schema correct |

| mixed\_brief\_json | 2 | 1.000 | Answer line + JSON schema correct |

| toolcall | 2 | 1.000 | Tool parse + schema valid |

| json | 4 | n/a | bucket\_score avg = 10.000 |

| fuzz | 20 | n/a | bucket\_score avg = 10.000 |

| mcq | 5 | n/a | bucket\_score avg = 1.000 |

Q4\_K\_M (`quantized_llama_cpp`)

|---|---:|---:|---:|---|

| json\_multistep | 5 | 0.600 | 0.000 | No degradation at Tier-1 |

| stateful\_followup | 2 | 1.000 | 0.000 | Perfect retention |

| toolcall\_only | 2 | 0.000 | −1.000 | ⚠️ Full degradation — see below |

| mixed\_brief\_json | 2 | 1.000 | 0.000 | No degradation |

| toolcall | 2 | 1.000 | 0.000 | No degradation |

| json | 4 | n/a | — | bucket\_score avg = 10.000 |

| fuzz | 20 | n/a | — | bucket\_score avg = 10.000 |

| mcq | 5 | n/a | — | bucket\_score avg = 1.000 |

⚠️ Quantization Degradation Finding — toolcall\_only

toolcall_only degraded from 1.000 (F16) to 0.000 (Q4\_K\_M). Both test cases failed on tool_name_ok and args_ok simultaneously. This means the quantized model, when asked to emit a bare tool-call JSON with no surrounding prose or chain-of-thought scaffold, lost the ability to produce a schema-valid payload entirely.

Practical implication: Do not deploy this Q4\_K\_M variant in pipelines where the model is expected to emit raw tool-call JSON without an external schema enforcement layer or retry loop. The toolcall family (tool call embedded in a broader response) remained at 1.000, indicating this degradation is specific to the strict schema-only output format.

This is exactly the kind of signal that pre-deployment quantization evaluation is designed to surface. The F16 baseline passes cleanly; the Q4\_K\_M variant does not. Without independent evaluation of both formats before deployment, this failure mode reaches production silently.

Signal-Level Diagnostics (Q4\_K\_M)

json\_multistep

| Signal | Rate | Tier |

|---|---:|---|

| schema\_ok | 1.000 | Tier-1 (gating) |

| checks\_consistent\_ok | 0.800 | Tier-1 (gating) |

| stop\_semantics\_ok | 1.000 | Tier-1 (gating) |

| oracle\_equiv\_ok | 0.600 | Tier-1 (gating) |

| final\_consistent\_ok | 0.000 | Tier-2 (tracked, non-gating) |

| final\_match\_reported | 0.000 | Tier-2 (tracked, non-gating) |

Note on Tier-2: final_consistent_ok and final_match_reported are not gating signals. Most deployed agentic systems compute and validate state externally; the model is not expected to report its own final state with exact fidelity. Tier-1 oracle equivalence (0.600) is the production-relevant signal here.

stateful\_followup

| Signal | Rate | Tier |

|---|---:|---|

| turn1\_parse\_ok | 1.000 | Tier-1 |

| turn2\_parse\_ok | 1.000 | Tier-1 |

| turn1\_exact\_match | 1.000 | Tier-1 |

| turn2\_exact\_match | 1.000 | Tier-1 |

toolcall\_only

| Signal | Rate | Tier |

|---|---:|---|

| tool\_name\_ok | 0.000 | Tier-1 |

| args\_ok | 0.000 | Tier-1 |

mixed\_brief\_json

| Signal | Rate | Tier |

|---|---:|---|

| answer\_line\_ok | 1.000 | Tier-1 |

| json\_parse\_ok | 1.000 | Tier-1 |

| schema\_ok | 1.000 | Tier-1 |

Recommended Use Cases

Derived from catalog_recommendation.json (quant_eval v7.21, run 20260209_170235).

✅ Deploy with Confidence (Q4\_K\_M)

Stateful multi-turn agents — Two-turn state retention is perfect (1.000). Suitable for conversational agents where turn-2 depends on turn-1 parsed state.
Structured JSON outputs (single-step) — bucket\_score avg of 10.000 on both json and fuzz families indicates consistently valid structured outputs.
Hybrid brief + JSON responses — mixed_brief_json passes at 1.000; combining a natural language answer line with a JSON payload is reliable.
Tool-calling with response scaffolding — toolcall (tool call embedded within a broader response) passes at 1.000. Use with a response template or instruction scaffold.
JSON multi-step with external validation loop — Pass rate of 0.600 is below the conservative PASS threshold but workable when an external planner or repair loop verifies each step.

⚠️ Use with Guardrails (Q4\_K\_M)

Bare tool-call dispatch (schema-only output) — toolcall_only failed completely (0.000). A schema enforcement layer, retry policy, or output parser is required for reliable tool dispatch without surrounding prose.

❌ Not Recommended (Q4\_K\_M)

Unassisted multi-step planning — Where planning correctness must hold without external verification or oracle validation.

Hardware Requirements

| Configuration | VRAM Required | Recommended GPU |

|---|---|---|

| Q4\_K\_M (this repo) · GPU only | ~10–12 GB | T4 16 GB · RTX 3080/4080 · A10G |

| Q4\_K\_M · CPU offload fallback | 8 GB VRAM + 4 GB RAM | Any CUDA-capable GPU |

| F16 baseline (companion repo) | ~30 GB | A100 40 GB · RTX 4090 · 2× A10G |

Usage

Installation

pip install llama-cpp-python huggingface_hub

For GPU acceleration (CUDA):

CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python --force-reinstall --no-cache-dir

Python — llama-cpp-python

from huggingface_hub import hf_hub_download
from llama_cpp import Llama

# Download directly from HuggingFace Hub
model_path = hf_hub_download(
    repo_id="pbhappliedsystems/ministral-3-14b-instruct-2512-gguf-Q4-K-M",
    filename="ministral-3-14b-instruct-2512-gguf-Q4-K-M.gguf"
)

llm = Llama(
    model_path=model_path,
    n_ctx=8192,          # context window; increase up to 32768 per model spec
    n_gpu_layers=-1,     # -1 offloads all layers to GPU
    verbose=False,
)

response = llm.create_chat_completion(
    messages=[
        {
            "role": "system",
            "content": "You are a helpful, concise assistant. Respond in structured JSON when asked."
        },
        {
            "role": "user",
            "content": "Summarize the following contract clause and flag any obligations: ..."
        }
    ],
    temperature=0.15,
    max_tokens=1024,
)

print(response["choices"][0]["message"]["content"])

For tool-calling use cases, enforce output schema externally (see evaluation findings above):

import json
from huggingface_hub import hf_hub_download
from llama_cpp import Llama

model_path = hf_hub_download(
    repo_id="pbhappliedsystems/ministral-3-14b-instruct-2512-gguf-Q4-K-M",
    filename="ministral-3-14b-instruct-2512-gguf-Q4-K-M.gguf"
)

llm = Llama(
    model_path=model_path,
    n_ctx=4096,
    n_gpu_layers=-1,
    verbose=False,
)

def call_with_tool_enforcement(prompt: str, retries: int = 3) -> dict:
    """
    Wrap tool-call dispatch with schema enforcement and retry.
    Required for Q4_K_M: toolcall_only pass rate = 0.000 (see eval above).
    """
    for attempt in range(retries):
        response = llm.create_chat_completion(
            messages=[
                {"role": "system", "content": "Respond only with a valid JSON tool call."},
                {"role": "user", "content": prompt}
            ],
            temperature=0.0,
            max_tokens=256,
        )
        raw = response["choices"][0]["message"]["content"].strip()
        try:
            parsed = json.loads(raw)
            assert "tool_name" in parsed and "args" in parsed
            return parsed
        except (json.JSONDecodeError, AssertionError):
            if attempt == retries - 1:
                raise ValueError(f"Tool call failed after {retries} attempts. Raw: {raw}")

result = call_with_tool_enforcement("Place item P on shelf A.")

CLI — llama-cli

# One-shot prompt
llama-cli \
  --model ministral-3-14b-instruct-2512-gguf-Q4-K-M.gguf \
  --chat-template mistral \
  --system-prompt "You are a helpful assistant." \
  --prompt "Summarize the following and return a JSON object with keys: summary, risk_level, action_items." \
  --n-predict 512 \
  --ctx-size 8192 \
  --n-gpu-layers -1 \
  --temp 0.15

For server deployment (OpenAI-compatible endpoint):

llama-server \
  --model ministral-3-14b-instruct-2512-gguf-Q4-K-M.gguf \
  --chat-template mistral \
  --ctx-size 8192 \
  --n-gpu-layers -1 \
  --port 8080 \
  --host 0.0.0.0

Then query via the OpenAI-compatible API:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-required")

response = client.chat.completions.create(
    model="ministral-3-14b-instruct-2512-gguf-Q4-K-M",
    messages=[{"role": "user", "content": "Your prompt here"}],
    temperature=0.15,
)
print(response.choices[0].message.content)

Artifact Provenance

|---|---|---|---|

Both artifacts were produced from mistralai/Ministral-3-14B-Instruct-2512 using a custom-built llama.cpp conversion and quantization pipeline developed by PBH Applied Systems. Conversion and quantization were performed on the full HuggingFace snapshot without modification to model weights prior to conversion.

Evaluation Methodology

quant_eval v7.21 is a proprietary behavioral evaluation harness developed by PBH Applied Systems. It evaluates both the full-precision (F16) and quantized variants of a model against an identical fixture set, enabling direct comparison of capability retention across quantization levels.

Fixture set: golden_oracle_fixtures_v7_21 (SHA256: 6d71a0b9147c079371b02a94f3c149eb78a6adc03dc16ff6833b964fbf4174f0)

Task families evaluated:

| Family | Description | Pass Signals |

|---|---|---|

| fuzz | Property-based regression; structured placement correctness | schema\_ok, constraints\_ok |

| json | Single-step structured JSON with constraint rules | schema\_ok, constraints\_ok |

| json_multistep | Multi-step planning with self-check and oracle verification | schema\_ok, checks\_consistent\_ok, stop\_semantics\_ok, oracle\_equiv\_ok |

| mcq | Multiple-choice extraction | choice\_ok |

| stateful_followup | Two-turn state tracking; turn-2 correct given turn-1 | turn1/2\_parse\_ok, turn1/2\_exact\_match |

| mixed_brief_json | Hybrid: natural language answer + valid JSON block | answer\_line\_ok, json\_parse\_ok, schema\_ok |

| toolcall | Tool call embedded in response; parse + schema validation | stage1\_tool\_parse\_ok, stage1\_tool\_schema\_ok |

| toolcall_only | Bare schema-only tool call; strict tool name + args check | tool\_name\_ok, args\_ok |

Scores are conservative conjunctions — a case passes only when all gating signals succeed. This is intentional: partial success in a deployed agent is often indistinguishable from failure at the system level.

Evaluation hardware: NVIDIA RTX 4090 (24 GB VRAM)

Evaluation date: February 9, 2026

quant_eval seed: 42

🔬 About quant_eval & This Evaluation Series

quant_eval is a proprietary behavioral evaluation harness developed by PBH Applied Systems, LLC. It measures real agent-adjacent task performance across structured output, tool dispatch, multi-turn state retention, and multi-step planning — not perplexity or leaderboard proxies. Every model published under pbhappliedsystems has been independently evaluated using quant_eval before being recommended for any production role.

See it in action: Live AI Agent Demo →

The demo runs production-style agent workflows powered by open-weight models selected through the quant_eval evaluation pipeline.

Need a deployment recommendation?

Not sure which quantization level is right for your hardware, latency target, or agent type?

→ pbhappliedsystems.com

Evaluated and published by PBH Applied Systems, LLC · patrick@pbhappliedsystems.com

About PBH Applied Systems

PBH Applied Systems, LLC is an Oklahoma City–based applied machine learning and AI systems company specializing in production-grade model evaluation, quantization pipelines, agentic AI infrastructure, and scalable AI-driven application development. The organization operates with a strong emphasis on engineering rigor, reproducibility, and real-world deployment constraints — particularly in environments where performance, cost efficiency, and reliability must be balanced against available hardware and budget.

Founder — Patrick Hill, M.S.

PBH Applied Systems was founded by Patrick Hill, a Data Scientist and AI/ML Engineer with 10+ years of experience delivering advanced analytics, predictive modeling, and decision-support solutions across high-stakes operational environments. Patrick holds a Master of Science in Software Engineering with concentrations in Artificial Intelligence and Machine Learning and a B.S. in Business Finance.

Technical expertise spans:

Languages & Data: Python, SQL, Linux, Pandas, NumPy, scikit-learn
ML & Modeling: Supervised and unsupervised learning, neural networks, NLP, transformers, regression, classification, forecasting, and feature engineering
AI/ML Frameworks: PyTorch, TensorFlow/Keras, HuggingFace Transformers, GGUF, llama.cpp, BitsAndBytes, PEFT, QLoRA
Deployment & MLOps: Flask APIs, Docker, CI/CD pipelines, REST endpoints, streaming inference, version control
Data Platforms: Jupyter, Databricks, Power BI, Matplotlib
Quantization: GGUF conversion, Q4\_K\_M / Q5\_K\_M / Q8\_0 strategies, adapter-per-model evaluation architecture

Published Author

Patrick is the author of Applied Machine Learning: Concepts, Tools, and Case Studies — a 1,200+ page practitioner-oriented textbook covering statistical modeling, supervised and unsupervised learning, neural networks, NLP, and real-world decision-support case studies. The text has been adopted as required reading for CSC 373 – Machine Learning at the University of Advancing Technology, and reflects the same philosophy applied across all PBH systems: prioritize practical correctness over theoretical novelty, favor interpretable and reliable solutions, and introduce complexity only when justified by data and deployment constraints.

Core Service Areas

1. LLM Optimization & Deployment

End-to-end conversion of full-weight HuggingFace models to production-ready GGUF format, with quantization strategies matched to target hardware and latency requirements. Custom-built llama.cpp pipelines with adapter-per-model architecture ensuring strict separation of concerns and universal cross-model compatibility.

2. AI Evaluation Frameworks

Proprietary behavioral evaluation via quant_eval — multi-run, timestamped pipelines producing structured artifacts, SHA256-verified manifests, per-family pass rates, F16 vs. quantized delta analysis, and deployment-ready recommendations. Evaluation batteries cover structured JSON output, multi-step reasoning, tool-calling fidelity, MCQ benchmarking, and fuzz/regression testing.

3. Agentic AI Infrastructure

Design and deployment of agent-oriented architectures using LlamaIndex ReAct agents, Flask orchestration layers, and serverless GPU inference. Full pipeline from model selection through quantization, evaluation, and production serving — including lead capture flows, budget controls, and API gateway integration.

4. Scalable AI Application Development

Production-grade multimodal AI applications integrating quantized LLMs, Whisper (speech-to-text), and BLIP (vision) via modular Flask APIs with Dockerized deployment and streaming-style responses. Advanced time-series forecasting systems featuring custom lightweight attention mechanisms, ensemble meta-learning, Bayesian hyperparameter optimization with resource-aware OOM backoff, and FinBERT sentiment fusion for hybrid structured/unstructured data pipelines.

5. ML Pipeline Design & Analytics

End-to-end data and model pipelines engineered for decision-support and operational forecasting. Encompasses feature engineering, leak-free forward-chaining cross-validation, KPI dashboard development, and analytical governance procedures designed for reproducibility at scale. Proven track record of translating complex model outputs into actionable insights for senior stakeholders across large-scale operational datasets.

6. Model & Agent Cataloging

Structured model catalog publishing with reproducible artifacts, standardized reporting, and clear performance tradeoff documentation — enabling engineering teams to make informed deployment decisions without re-running evaluations from scratch.

Engineering Principles

Reproducibility first — Every run produces structured artifacts, versioned manifests, and comparable outputs
Universality as a requirement — Systems work across models without custom rewrites per deployment
No silent behavior changes — Evaluation logic, prompts, and workflows are locked and versioned
GPU utilization is non-negotiable — All pipelines are designed to fully leverage available hardware
Separation of IP and operations — Core intellectual property is maintained independently of client deliverables

📞 Work With PBH Applied Systems

The toolcall_only degradation documented in this card — 1.000 (F16) → 0.000 (Q4\_K\_M) — is a representative example of what rigorous pre-deployment evaluation surfaces. This finding is invisible to perplexity scores, benchmark leaderboards, and casual manual testing. It only appears when you run the model against structured behavioral tasks under production-equivalent conditions.

If you are selecting, deploying, or building on quantized open-weight models, evaluation like this belongs in your deployment process.

👉 Book a Scoping Call — Discuss your model selection, quantization strategy, or deployment architecture directly with Patrick.

👉 Request an Evaluation Report — A full quant_eval behavioral audit for your target model(s): per-family pass rates, F16 vs. quantized delta analysis, failure cluster diagnostics, and a deployment recommendation. Engagements from $2,500.

Connect

| | |

|---|---|

| 🌐 Website | pbhappliedsystems.com |

| 📧 Email | patrick@pbhappliedsystems.com |

| 💼 LinkedIn | PBH Applied Systems, LLC |

| ▶️ YouTube | @pbhappliedsystems |

| 📸 Instagram | @pbhappliedsystems |

| 👍 Facebook | pbhappliedsystems |

License

This GGUF repository inherits the license of the base model:

Apache 2.0 — mistralai/Ministral-3-14B-Instruct-2512

The quant_eval evaluation methodology, fixture set, and scoring framework are proprietary to PBH Applied Systems, LLC and are not included in this repository.

GGUF conversion, quantization, and behavioral evaluation performed by PBH Applied Systems, LLC · quant_eval v7.21 · Run ID: 20260209_170235

pbhappliedsystems/ministral-3-14b-instruct-2512-gguf-Q4-K-M

Ministral-3-14B-Instruct-2512 · GGUF Q4\_K\_M

Try This Model in the Live AI Agent Demo

Model Description

Key Characteristics

PBH Applied Systems Evaluation — quant\_eval v7.21

Aggregate Scores (Q4\_K\_M)

Per-Family Pass Rates

F16 Baseline (full_weight_transformers)

Q4\_K\_M (quantized_llama_cpp)

⚠️ Quantization Degradation Finding — toolcall\_only

Signal-Level Diagnostics (Q4\_K\_M)

json\_multistep

stateful\_followup

toolcall\_only

mixed\_brief\_json

Recommended Use Cases

✅ Deploy with Confidence (Q4\_K\_M)

⚠️ Use with Guardrails (Q4\_K\_M)

❌ Not Recommended (Q4\_K\_M)

Hardware Requirements

Usage

Installation

Python — llama-cpp-python

CLI — llama-cli

Artifact Provenance

Evaluation Methodology

🔬 About quant_eval & This Evaluation Series

About PBH Applied Systems

Founder — Patrick Hill, M.S.

Published Author

Core Service Areas

Engineering Principles

📞 Work With PBH Applied Systems

Connect

License

F16 Baseline (`full_weight_transformers`)

Q4\_K\_M (`quantized_llama_cpp`)