Mistral-Nemo-Instruct-2407 · GGUF Q4\_K\_M

Quantized, converted, and evaluated by PBH Applied Systems, LLC

— Applied AI/ML Consulting · LLM Optimization & Deployment · Quantized AI Infrastructure

🔬 This repository is part of a production-oriented evaluation series. Every model published under pbhappliedsystems has been independently evaluated using quant_eval v7.21 — a proprietary behavioral evaluation harness developed by PBH Applied Systems. Scores measure real agent-adjacent task performance across structured output, tool dispatch, multi-turn state retention, and multi-step planning families — not perplexity or benchmark leaderboard proxies.

Try This Model in the Live AI Agent Demo

Launch the PBH Applied Systems AI Agent Demo →

This model is part of the PBH Applied Systems live AI Agent Demo, where visitors can test evaluated quantized open-weight models across production-style agent workflows: reasoning and analysis, document intelligence, and code automation.

The demo uses quant_eval results to show how model selection changes by task. A model that performs well for long-context document analysis may not be the best choice for hard multi-step planning, strict tool-use workflows, or production code generation. Each deployed model is evaluated for practical agent behavior, including coherence, instruction following, reasoning, task completion, structured output reliability, tool-use behavior, and quantization impact.

For this repository, the Q4_K_M variant represents the deployment-focused model: smaller, faster, and more cost-efficient than the F16 baseline. The evaluation results below explain where this quantized model preserves useful behavior, where quantization introduces risk, and what guardrails are recommended before production deployment.

The purpose of the demo is simple: let prospects test the same kind of evaluated quantized models that PBH Applied Systems deploys for real agentic AI systems.

Model Description

This repository contains the 4-bit quantized (Q4\_K\_M) GGUF of mistralai/Mistral-Nemo-Instruct-2407, a 12-billion parameter instruction-tuned model developed by Mistral AI in collaboration with NVIDIA (July 2024 release). Mistral-Nemo features the Tekken tokenizer and a 128,000-token context window — the largest context in the evaluated series, outside of the Qwen2.5-14B-Instruct-1M, which supports a 1 million token window, in the PBH Applied Systems evaluated series.

The Q4\_K\_M format applies 4-bit quantization with K-quant medium precision. As documented in the evaluation section below, Q4\_K\_M quantization produces measurable degradation on multi-step planning tasks — including a complete breakdown on the hardest planning case — while preserving strong performance on stateful, structured-output, and hybrid response tasks.

The full-precision F16 baseline is published separately at pbhappliedsystems/mistral-nemo-instruct-2407-gguf-F16.

Key Characteristics

Parameters: 12B
Format: GGUF Q4\_K\_M
File size: 7.48 GB
SHA256: 5765024ff3361f6dc5b590b963b378bd2e87ac95eabe5823a08a3ad336b498c9
Context window: 128,000 tokens (Tekken tokenizer)
Minimum VRAM (GPU inference): ~10 GB (T4 class or better)
Recommended GPU tier: NVIDIA T4 (16 GB) · RTX 3080/4080 · A10G
Inference speed (eval hardware): avg 1.42 sec/case on RTX 4090
Multilingual: English, French, German, Spanish, Italian, Portuguese, Russian, Chinese, Japanese

PBH Applied Systems Evaluation — quant\_eval v7.21

Evaluation conducted by PBH Applied Systems, LLC using quant_eval v7.21

Run ID: 20260211_022944 · Fixtures: golden_oracle_fixtures_v7_21 (SHA256: 6d71a0b9147c...) · Seed: 42

Hardware: NVIDIA RTX 4090 · Total rows evaluated: 84 (42 F16 · 42 Q4\_K\_M)

Aggregate Scores (Q4\_K\_M)

Scores are normalized to [0.0 – 1.0]. Higher is better.

| Dimension | Score |

|---|---:|

| Task Completion | 0.6631 |

| Reasoning | 0.7870 |

| Coherence | 0.8836 |

| Instruction Following | 0.9329 |

| Avg inference time | 1.42 sec/case |

Per-Family Pass Rates

F16 Baseline (`full_weight_transformers`)

|---|---:|---:|---:|---|

| json\_multistep | 5 | 0.600 | 79.08 | ms\_easy\_02 + ms\_hard\_01 fail |

| stateful\_followup | 2 | 1.000 | 7.60 | Both turns exact match |

| toolcall\_only | 2 | 1.000\* | 9.54 | Gating passed; schema wrapper issue — see note |

| mixed\_brief\_json | 2 | 1.000 | 10.59 | Answer line + JSON schema correct |

| toolcall | 2 | 1.000 | 14.39 | Tool parse + schema valid |

| json | 4 | n/a | 64.02 | bucket\_score avg = 10.000 (json\_01 = 149.88s outlier) |

| fuzz | 20 | n/a | 26.61 | bucket\_score avg = 10.000 |

| mcq | 5 | n/a | 0.46 | bucket\_score avg = 0.600 — 2 failures |

Q4\_K\_M (`quantized_llama_cpp`)

|---|---:|---:|---:|---:|---|

| json\_multistep | 5 | 0.400 | −0.200 | 4.52 | ⚠️ Hard case complete breakdown |

| stateful\_followup | 2 | 1.000 | 0.000 | 0.40 | Perfect retention |

| toolcall\_only | 2 | 0.000 | −1.000 | 0.46 | tool\_name\_ok=1, args\_ok=0 |

| mixed\_brief\_json | 2 | 1.000 | 0.000 | 0.52 | No degradation |

| toolcall | 2 | 1.000 | 0.000 | 0.62 | Pass rate holds; final\_mismatch on tool\_02 — see note |

| json | 4 | n/a | — | 1.65 | bucket\_score avg = 10.000 |

| fuzz | 20 | n/a | — | 1.31 | bucket\_score avg = 10.000 |

| mcq | 5 | n/a | — | 0.03 | bucket\_score avg = 0.400 — 3 failures |

Key Findings

Finding 1: json\_multistep — Complete Breakdown on Hard Case

The drop from 0.600 (F16) to 0.400 (Q4\_K\_M) is the largest json\_multistep degradation in the PBH Applied Systems evaluated series to date. The case-level breakdown reveals why:

|---|---|---|---|---|

ms\_hard\_01 at Q4\_K\_M is a total failure: schema_ok=0, checks_consistent_ok=0, stop_semantics_ok=0, oracle_equiv_ok=0. All four Tier-1 gating signals fail simultaneously. The model does not produce a parseable schema response, its intermediate checks are self-inconsistent, its STOP semantics are wrong, and the computed final state does not match the oracle. The F16 variant fails this case too — but only on consistency and oracle, not on schema or STOP semantics. Quantization turns a partial failure into a complete one.

ms\_med\_02 is a new failure at Q4\_K\_M that passes at F16: checks_consistent_ok=0 with oracle_equiv_ok=1 — the model arrives at the correct final state but its internal reasoning steps are self-inconsistent. This is a structural coherence regression under quantization.

Practical implication: This model at Q4\_K\_M should not be used for multi-step planning tasks without an external validation layer. The hard case cannot be considered reliably solvable at this precision level.

Finding 2: toolcall — Final Mismatch on tool\_02 (Q4\_K\_M)

toolcall passes at 1.000 (both stage1 signals pass), but tool_02 shows detail=final_mismatch with bucket_score=0 at Q4\_K\_M. The tool call JSON is dispatched and validated correctly — the stage-1 parse and schema check both pass — but the model's final answer (the computed result returned after tool execution) does not match the expected output.

|---|---:|---:|---|

| tool\_01 | 11 | 11 | ok |

| tool\_02 | 11 | 0 | final\_mismatch |

This is not a gating failure — the pass rate remains 1.000 because Tier-1 only evaluates the dispatch quality, not the final answer. However, in a production pipeline where tool results feed downstream computation, a final\_mismatch means the model called the tool correctly but gave a wrong answer when reporting the result. For applications where post-tool reasoning accuracy matters, treat this as a deployment risk at Q4\_K\_M.

The Q4\_K\_M toolcall bucket\_score average of 5.5 (vs 11.0 at F16) directly reflects this: one perfect (11) and one complete failure (0) averaged together.

Finding 3: toolcall\_only — Consistent args Failure with Stable Tool Name

| Signal | F16 Rate | Q4\_K\_M Rate |

|---|---:|---:|

| tool\_name\_ok | 1.000 | 1.000 |

| args\_ok | 1.000 | 0.000 |

| schema\_ok | 0.000\* | 0.000 |

At F16, toolcall_only passes gating (tool\_name\_ok=1, args\_ok=1) but carries the same schema wrapper non-compliance observed across multiple models in this series (schema_ok=0, detail=schema_error). At Q4\_K\_M, tool_name_ok stays perfect at 1.000 — the model correctly identifies which tool to call — but args_ok drops to 0.000 on both cases. The quantized model knows the tool name but cannot construct a valid argument payload.

\*F16 schema\_ok=0 is a non-gating wrapper issue (uses "tool" instead of "tool_name" as outer key), not a capability failure. Both gating signals pass at F16.

Finding 4: MCQ "got=A" Bias

Both runners show a systematic bias toward selecting choice A on failures:

| Case | F16 result | Q4\_K\_M result |

|---|---|---|

| mcq\_01 | ✅ ok | ✅ ok |

| mcq\_02 | ❌ wrong\_choice got=A | ❌ wrong\_choice got=A |

| mcq\_03 | ✅ ok | ✅ ok |

| mcq\_04 | ✅ ok | ❌ wrong\_choice got=A |

| mcq\_05 | ❌ wrong\_choice got=A | ❌ wrong\_choice got=A |

Every failure on both runners produces got=A. This is a model-level characteristic: when uncertain, this model defaults to option A. Q4\_K\_M extends this bias to mcq\_04 (which F16 answers correctly), reducing the bucket\_score from 0.600 to 0.400. For MCQ applications, be aware of this A-default tendency and consider instruction-tuning or chain-of-thought prompting to elicit more deliberate choice selection.

Signal-Level Diagnostics (Q4\_K\_M)

json\_multistep

|---|---:|---:|---:|

| schema\_ok | 1.000 | 0.800 | −0.200 |

| checks\_consistent\_ok | 0.800 | 0.400 | −0.400 |

| stop\_semantics\_ok | 1.000 | 0.800 | −0.200 |

| oracle\_equiv\_ok | 0.600 | 0.600 | 0.000 |

| final\_consistent\_ok | 0.000 | 0.000 | 0.000 |

| final\_match\_reported | 0.000 | 0.000 | 0.000 |

checks_consistent_ok takes the largest hit (−0.400), dropping from 0.800 to 0.400. This signal measures whether the model's intermediate reasoning steps are internally self-consistent. The Q4\_K\_M degradation here is the root cause of the json\_multistep pass rate drop: the model fails ms\_med\_02 on consistency alone, and ms\_hard\_01 on all signals.

stateful\_followup

| Signal | Rate |

|---|---:|

| turn1\_parse\_ok | 1.000 |

| turn2\_parse\_ok | 1.000 |

| turn1\_exact\_match | 1.000 |

| turn2\_exact\_match | 1.000 |

toolcall\_only (Q4\_K\_M)

| Signal | Rate |

|---|---:|

| tool\_name\_ok | 1.000 |

| args\_ok | 0.000 |

mixed\_brief\_json

| Signal | Rate |

|---|---:|

| answer\_line\_ok | 1.000 |

| json\_parse\_ok | 1.000 |

| schema\_ok | 1.000 |

Recommended Use Cases

✅ Deploy with Confidence (Q4\_K\_M)

Stateful multi-turn agents — Perfect two-turn state retention (1.000). Reliable at 0.40 sec/case.
Structured JSON outputs (single-step) — json and fuzz both achieve bucket\_score 10.000. Valid constraint-adherent outputs every case.
Hybrid brief + JSON responses — mixed_brief_json passes at 1.000. Fast at 0.52 sec/case.
Long-context document processing — 128K token context window is the key differentiator for this model. Suitable for full-document Q&A, multi-document comparison, and long-form extraction tasks.
Multilingual structured tasks — Trained on 9 languages. Reliable for non-English structured output pipelines.
Tool-calling with response scaffolding — toolcall pass rate holds at 1.000. Use stage-1 tool dispatch reliably; add final-answer validation for downstream computation (see tool\_02 finding).

⚠️ Use with Guardrails (Q4\_K\_M)

Multi-step planning at easy-to-medium difficulty — ms\_easy\_01 and ms\_med\_01 pass. ms\_med\_02 fails on internal consistency. Use with an external validation loop for any planning task beyond trivial difficulty.
Post-tool answer validation — toolcall dispatches correctly but tool\_02 returns a wrong final answer. Validate model output after tool execution, not just the tool call itself.
Bare tool-call dispatch — toolcall_only fails on args (0.000). Add a schema enforcement layer or use scaffolded tool calling.

❌ Not Recommended (Q4\_K\_M)

Hard multi-step planning — ms\_hard\_01 fails on all four gating signals simultaneously. Do not deploy for hard planning tasks without F16 or an external planner/verifier.
MCQ without A-bias mitigation — Three of five MCQ cases fail at Q4\_K\_M, all defaulting to A. Add chain-of-thought prompting or answer validation for MCQ-style pipelines.

128K Context Window — Deployment Considerations

Mistral-Nemo's 128K context window is a meaningful production advantage for document-intensive applications. At Q4\_K\_M (7.48 GB model weight), actual usable context depends on KV cache VRAM overhead:

|---|---|---|---|

| 8K tokens | ~0.5 GB | ~10 GB | T4 16 GB |

| 32K tokens | ~2 GB | ~12 GB | T4 16 GB · RTX 4080 |

| 64K tokens | ~4 GB | ~14 GB | A10G 24 GB · RTX 4090 |

| 128K tokens | ~8 GB | ~18 GB | A10G 24 GB · RTX 4090 |

Set n_ctx in llama-cpp-python to the actual context length you need — do not default to 128K if your use case only needs 8K. Unnecessary context allocation wastes VRAM and slows inference.

Hardware Requirements

| Configuration | VRAM Required | Recommended GPU |

|---|---|---|

| Q4\_K\_M · 8K context (this repo) | ~10 GB | T4 16 GB · RTX 3080 |

| Q4\_K\_M · 128K context | ~18 GB | A10G 24 GB · RTX 4090 |

| F16 baseline (companion repo) | ~26 GB | A100 40 GB · RTX 4090 · 2× A10G |

Usage

Installation

pip install llama-cpp-python huggingface_hub

For GPU acceleration (CUDA):

CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python --force-reinstall --no-cache-dir

Python — llama-cpp-python

from huggingface_hub import hf_hub_download
from llama_cpp import Llama

model_path = hf_hub_download(
    repo_id="pbhappliedsystems/mistral-nemo-instruct-2407-gguf-Q4-K-M",
    filename="mistral-nemo-instruct-2407-gguf-Q4-K-M.gguf"
)

llm = Llama(
    model_path=model_path,
    n_ctx=32768,      # Adjust to your use case; model supports up to 128K
    n_gpu_layers=-1,  # -1 offloads all layers to GPU
    verbose=False,
)

response = llm.create_chat_completion(
    messages=[
        {
            "role": "system",
            "content": "You are a precise assistant. Follow instructions exactly and return structured outputs when requested."
        },
        {
            "role": "user",
            "content": "Analyze the following document and return a JSON object with keys: summary, key_entities, sentiment, action_items."
        }
    ],
    temperature=0.3,
    max_tokens=1024,
)

print(response["choices"][0]["message"]["content"])

For long-document use (leveraging the 128K context window):

# Load a large document and process it within a single context window
with open("large_document.txt", "r") as f:
    document = f.read()

llm_long = Llama(
    model_path=model_path,
    n_ctx=65536,      # 64K context — adjust based on available VRAM
    n_gpu_layers=-1,
    verbose=False,
)

response = llm_long.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are a document analysis assistant."},
        {"role": "user", "content": f"Summarize the following document and extract all action items:\n\n{document}"}
    ],
    temperature=0.3,
    max_tokens=2048,
)
print(response["choices"][0]["message"]["content"])

For tool-calling with post-tool answer validation (addresses tool\_02 final\_mismatch finding):

import json
from huggingface_hub import hf_hub_download
from llama_cpp import Llama

model_path = hf_hub_download(
    repo_id="pbhappliedsystems/mistral-nemo-instruct-2407-gguf-Q4-K-M",
    filename="mistral-nemo-instruct-2407-gguf-Q4-K-M.gguf"
)

llm = Llama(model_path=model_path, n_ctx=4096, n_gpu_layers=-1, verbose=False)

def execute_tool(tool_name: str, args: dict) -> str:
    """Stub: replace with actual tool execution."""
    if tool_name == "add":
        return str(args["a"] + args["b"])
    raise ValueError(f"Unknown tool: {tool_name}")

def call_with_tool_and_validate(prompt: str) -> dict:
    """
    Scaffolded tool dispatch with post-execution answer validation.
    Addresses quant_eval v7.21 finding: tool_02 final_mismatch at Q4_K_M.
    toolcall stage1 pass rate = 1.000; final answer accuracy is not guaranteed.
    """
    response = llm.create_chat_completion(
        messages=[
            {
                "role": "system",
                "content": "You are a tool-calling assistant. Emit a tool call JSON, then report the result."
            },
            {"role": "user", "content": prompt}
        ],
        temperature=0.0,
        max_tokens=512,
    )
    raw = response["choices"][0]["message"]["content"]

    # Extract tool call
    import re
    match = re.search(r'\{[^{}]*"tool_name"[^{}]*\}', raw, re.DOTALL)
    if not match:
        raise ValueError(f"No tool call found: {raw[:200]}")
    call = json.loads(match.group(0))

    # Execute tool independently — do not trust model's reported result
    actual_result = execute_tool(call["tool_name"], call["args"])
    return {"tool_call": call, "validated_result": actual_result, "model_raw": raw}

result = call_with_tool_and_validate("What is 10 minus 4?")
print(f"Validated result: {result['validated_result']}")

CLI — llama-cli

# One-shot prompt
llama-cli \
  --model mistral-nemo-instruct-2407-gguf-Q4-K-M.gguf \
  --chat-template mistral \
  --system-prompt "You are a precise assistant." \
  --prompt "Analyze the following and return a JSON object with keys: summary, risk_level, action_items." \
  --n-predict 1024 \
  --ctx-size 32768 \
  --n-gpu-layers -1 \
  --temp 0.3

For server deployment (OpenAI-compatible endpoint):

llama-server \
  --model mistral-nemo-instruct-2407-gguf-Q4-K-M.gguf \
  --chat-template mistral \
  --ctx-size 32768 \
  --n-gpu-layers -1 \
  --port 8080 \
  --host 0.0.0.0

Query via the OpenAI-compatible API:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-required")

response = client.chat.completions.create(
    model="mistral-nemo-instruct-2407-gguf-Q4-K-M",
    messages=[{"role": "user", "content": "Your prompt here"}],
    temperature=0.3,
)
print(response.choices[0].message.content)

Artifact Provenance

|---|---|---|---|

Both artifacts were produced from mistralai/Mistral-Nemo-Instruct-2407 using a custom-built llama.cpp conversion and quantization pipeline developed by PBH Applied Systems.

Evaluation Methodology

quant_eval v7.21 is a proprietary behavioral evaluation harness developed by PBH Applied Systems. It evaluates both the full-precision (F16) and quantized variants against an identical fixture set, enabling direct comparison of capability retention across quantization levels.

Fixture set: golden_oracle_fixtures_v7_21 (SHA256: 6d71a0b9147c079371b02a94f3c149eb78a6adc03dc16ff6833b964fbf4174f0)

| Family | Description | Pass Signals |

|---|---|---|

| fuzz | Property-based regression; structured placement correctness | schema\_ok, constraints\_ok |

| json | Single-step structured JSON with constraint rules | schema\_ok, constraints\_ok |

| json_multistep | Multi-step planning with self-check and oracle verification | schema\_ok, checks\_consistent\_ok, stop\_semantics\_ok, oracle\_equiv\_ok |

| mcq | Multiple-choice extraction | choice\_ok |

| stateful_followup | Two-turn state tracking; turn-2 correct given turn-1 | turn1/2\_parse\_ok, turn1/2\_exact\_match |

| mixed_brief_json | Hybrid: natural language answer + valid JSON block | answer\_line\_ok, json\_parse\_ok, schema\_ok |

| toolcall | Tool call embedded in response; parse + schema validation | stage1\_tool\_parse\_ok, stage1\_tool\_schema\_ok |

| toolcall_only | Bare schema-only tool call; strict tool name + args check | tool\_name\_ok, args\_ok |

Evaluation hardware: NVIDIA RTX 4090 (24 GB VRAM)

Evaluation date: February 11, 2026

quant_eval seed: 42

🔬 About quant_eval & This Evaluation Series

quant_eval is a proprietary behavioral evaluation harness developed by PBH Applied Systems, LLC. It measures real agent-adjacent task performance across structured output, tool dispatch, multi-turn state retention, and multi-step planning — not perplexity or leaderboard proxies. Every model published under pbhappliedsystems has been independently evaluated using quant_eval before being recommended for any production role.

See it in action: Live AI Agent Demo →

The demo runs production-style agent workflows powered by open-weight models selected through the quant_eval evaluation pipeline.

Need a deployment recommendation?

Not sure which quantization level is right for your hardware, latency target, or agent type?

→ pbhappliedsystems.com

Evaluated and published by PBH Applied Systems, LLC · patrick@pbhappliedsystems.com

About PBH Applied Systems

PBH Applied Systems, LLC is an Oklahoma City–based applied machine learning and AI systems company specializing in production-grade model evaluation, quantization pipelines, agentic AI infrastructure, and scalable AI-driven application development. The organization operates with a strong emphasis on engineering rigor, reproducibility, and real-world deployment constraints — particularly in environments where performance, cost efficiency, and reliability must be balanced against available hardware and budget.

Founder — Patrick Hill, M.S.

PBH Applied Systems was founded by Patrick Hill, a Data Scientist and AI/ML Engineer with 10+ years of experience delivering advanced analytics, predictive modeling, and decision-support solutions across high-stakes operational environments. Patrick holds a Master of Science in Software Engineering with concentrations in Artificial Intelligence and Machine Learning and a B.S. in Business Finance.

Technical expertise spans:

Languages & Data: Python, SQL, Linux, Pandas, NumPy, scikit-learn
ML & Modeling: Supervised and unsupervised learning, neural networks, NLP, transformers, regression, classification, forecasting, and feature engineering
AI/ML Frameworks: PyTorch, TensorFlow/Keras, HuggingFace Transformers, GGUF, llama.cpp, BitsAndBytes, PEFT, QLoRA
Deployment & MLOps: Flask APIs, Docker, CI/CD pipelines, REST endpoints, streaming inference, version control
Data Platforms: Jupyter, Databricks, Power BI, Matplotlib
Quantization: GGUF conversion, Q4\_K\_M / Q5\_K\_M / Q8\_0 strategies, adapter-per-model evaluation architecture

Published Author

Patrick is the author of Applied Machine Learning: Concepts, Tools, and Case Studies — a 1,200+ page practitioner-oriented textbook adopted as required reading for CSC 373 – Machine Learning at the University of Advancing Technology.

Core Service Areas

1. LLM Optimization & Deployment — End-to-end GGUF conversion and quantization with custom llama.cpp pipelines and adapter-per-model architecture.

2. AI Evaluation Frameworks — Proprietary behavioral evaluation via quant_eval: per-family pass rates, F16 vs. quantized delta analysis, failure cluster diagnostics, deployment recommendations.

3. Agentic AI Infrastructure — LlamaIndex ReAct agents, Flask orchestration, serverless GPU inference, full pipeline from model selection to production serving.

4. Scalable AI Application Development — Multimodal applications (quantized LLMs + Whisper + BLIP), Dockerized Flask APIs, advanced time-series forecasting with custom attention mechanisms, Bayesian hyperparameter optimization, and FinBERT sentiment fusion.

5. ML Pipeline Design & Analytics — Feature engineering, forward-chaining cross-validation, KPI dashboards, analytical governance at scale.

6. Model & Agent Cataloging — Structured catalog publishing with reproducible artifacts and clear performance tradeoff documentation.

📞 Work With PBH Applied Systems

The complete breakdown of ms\_hard\_01 at Q4\_K\_M — all four gating signals failing simultaneously — and the tool\_02 final\_mismatch are findings that only appear when you run both the F16 and quantized variant against the same behavioral test suite. Neither shows up in standard benchmarks. Neither is visible from casual testing. Both have direct consequences for production deployment decisions.

A model that dispatches tools correctly but gives wrong answers, and that fails completely on hard planning cases, needs to be known before it goes to production — not after.

👉 Book a Scoping Call — Discuss your model selection, quantization strategy, or deployment architecture directly with Patrick.

👉 Request an Evaluation Report — A full quant_eval behavioral audit: per-family pass rates, F16 vs. quantized delta analysis, failure cluster diagnostics, and a deployment recommendation. Engagements from $2,500.

Connect

| | |

|---|---|

| 🌐 Website | pbhappliedsystems.com |

| 📧 Email | patrick@pbhappliedsystems.com |

| 💼 LinkedIn | PBH Applied Systems, LLC |

| ▶️ YouTube | @pbhappliedsystems |

| 📸 Instagram | @pbhappliedsystems |

| 👍 Facebook | pbhappliedsystems |

License

This GGUF repository inherits the license of the base model:

Apache 2.0 — mistralai/Mistral-Nemo-Instruct-2407

The quant_eval evaluation methodology, fixture set, and scoring framework are proprietary to PBH Applied Systems, LLC and are not included in this repository.

GGUF conversion, quantization, and behavioral evaluation performed by PBH Applied Systems, LLC · quant_eval v7.21 · Run ID: 20260211_022944

pbhappliedsystems/mistral-nemo-instruct-2407-gguf-Q4-K-M

Mistral-Nemo-Instruct-2407 · GGUF Q4\_K\_M

Try This Model in the Live AI Agent Demo

Model Description

Key Characteristics

PBH Applied Systems Evaluation — quant\_eval v7.21

Aggregate Scores (Q4\_K\_M)

Per-Family Pass Rates

F16 Baseline (full_weight_transformers)

Q4\_K\_M (quantized_llama_cpp)

Key Findings

Finding 1: json\_multistep — Complete Breakdown on Hard Case

Finding 2: toolcall — Final Mismatch on tool\_02 (Q4\_K\_M)

Finding 3: toolcall\_only — Consistent args Failure with Stable Tool Name

Finding 4: MCQ "got=A" Bias

Signal-Level Diagnostics (Q4\_K\_M)

json\_multistep

stateful\_followup

toolcall\_only (Q4\_K\_M)

mixed\_brief\_json

Recommended Use Cases

✅ Deploy with Confidence (Q4\_K\_M)

⚠️ Use with Guardrails (Q4\_K\_M)

❌ Not Recommended (Q4\_K\_M)

128K Context Window — Deployment Considerations

Hardware Requirements

Usage

Installation

Python — llama-cpp-python

CLI — llama-cli

Artifact Provenance

Evaluation Methodology

🔬 About quant_eval & This Evaluation Series

About PBH Applied Systems

Founder — Patrick Hill, M.S.

Published Author

Core Service Areas

📞 Work With PBH Applied Systems

Connect

License

F16 Baseline (`full_weight_transformers`)

Q4\_K\_M (`quantized_llama_cpp`)