Mistral-Nemo-Instruct-2407 · GGUF F16

Converted and evaluated by PBH Applied Systems, LLC

— Applied AI/ML Consulting · LLM Optimization & Deployment · Quantized AI Infrastructure

🔬 This repository is part of a production-oriented evaluation series. Every model published under pbhappliedsystems has been independently evaluated using quant_eval v7.21 — a proprietary behavioral evaluation harness developed by PBH Applied Systems. Scores measure real agent-adjacent task performance across structured output, tool dispatch, multi-turn state retention, and multi-step planning families — not perplexity or benchmark leaderboard proxies.

📌 This is the full-precision F16 baseline repository. The evaluated Q4\_K\_M deployment variant is published at pbhappliedsystems/mistral-nemo-instruct-2407-gguf-Q4-K-M. That card documents the full F16 vs. Q4\_K\_M comparison — including the json\_multistep degradation (0.600 → 0.400), a complete ms\_hard\_01 breakdown at Q4\_K\_M, and a tool\_02 final\_mismatch finding that does not appear at F16.

Try the Live AI Agent Demo

Launch the PBH Applied Systems AI Agent Demo →

This model is part of the PBH Applied Systems evaluated model series that supports the live AI Agent Demo. The demo lets visitors interact with production-style agent workflows powered by open-weight language models evaluated through PBH Applied Systems' quant_eval framework.

The F16 model serves a different role than the Q4_K_M deployment variant. F16 is the full-precision baseline used to measure what the model can do before quantization. quant_eval then compares the quantized model against this baseline to identify which capabilities are preserved, which degrade, and which tasks require guardrails or a higher-precision deployment.

This comparison is central to the demo. It helps determine which model belongs in which agent role:

Reasoning models are selected for planning, analysis, and auditable decision workflows.
Document models are selected for long-context extraction, summarization, and structured Q&A.
Code models are selected for task completion, structured output, API scaffolding, and automation workflows.
Quantized variants are selected when they preserve enough behavior to reduce cost, latency, and GPU requirements.
F16 variants remain important when maximum fidelity, cleaner tool execution, or reduced quantization risk matters more than speed or cost.

The live demo shows the deployment side of that process. The F16 card documents the reference behavior. The Q4_K_M card shows what changes after compression. Together, they explain how PBH Applied Systems uses quant_eval to choose the correct LLM for the correct agent type instead of guessing from model size or leaderboard reputation.

Model Description

This repository contains the full-precision F16 GGUF of mistralai/Mistral-Nemo-Instruct-2407, a 12-billion parameter instruction-tuned model developed by Mistral AI in collaboration with NVIDIA (July 2024 release). Mistral-Nemo features the Tekken tokenizer and a 128,000-token context window — the largest context in the PBH Applied Systems evaluated series outside of the Qwen2.5-14B-Instruct-1M, which supports a 1 million token window.

The F16 format preserves all original float16 weights without quantization. In the PBH Applied Systems evaluation pipeline, this F16 run (20260211_014604) served as the baseline cache generation pass — producing the full_weight_cache.json used as the reference anchor for the subsequent Q4\_K\_M comparison run (20260211_022944).

For most production deployments, the Q4\_K\_M variant is the appropriate choice. The F16 is the right choice when maximum output fidelity, clean tool-execution pipelines, and the full 128K context window at maximum precision are required.

Key Characteristics

Parameters: 12B
Format: GGUF F16 (full precision)
File size: 24.5 GB
SHA256: cc7b8c5c3f129ad32aee562017b5e56f1284ee7d92445be292c644a42b3c9556
Context window: 128,000 tokens (Tekken tokenizer)
Minimum VRAM (GPU inference): ~26 GB
Recommended GPU tier: A100 40 GB · RTX 4090 (24 GB, with offload) · 2× A10G
Inference speed (eval hardware): avg 30.24 sec/case on RTX 4090
Multilingual: English, French, German, Spanish, Italian, Portuguese, Russian, Chinese, Japanese

Why 30.24 sec/case avg? This is a 12B non-reasoning model at full F16 precision. The json family averaged 64.02 sec/case due to multi-step token generation, with json\_01 reaching 149.88 seconds — a significant single-case outlier visible in both F16 and Q4\_K\_M runs on this model. The Q4\_K\_M variant averages 1.42 sec/case (21.3× faster).

PBH Applied Systems Evaluation — quant\_eval v7.21

Evaluation conducted by PBH Applied Systems, LLC using quant_eval v7.21

Run ID: 20260211_014604 · Fixtures: golden_oracle_fixtures_v7_21 (SHA256: 6d71a0b9147c...) · Seed: 42

Hardware: NVIDIA RTX 4090 · Runner: full_weight_transformers (F16 only) · Total rows: 42

Note on aggregate scores: The normalized aggregate dimensions (task completion, reasoning, coherence, instruction following) are computed from the combined F16 + Q4\_K\_M comparison run and are reported on the Q4\_K\_M card. This F16 card reports per-family pass rates from the full_weight_transformers runner.

Per-Family Pass Rates — F16 (`full_weight_transformers`)

| Family | N | Pass Rate | Avg Secs | Min | Max | Notes |

|---|---:|---:|---:|---:|---:|---|

| json\_multistep | 5 | 0.600 | 79.08 | 53.91 | 92.74 | ms\_easy\_02 + ms\_hard\_01 fail |

| stateful\_followup | 2 | 1.000 | 7.60 | 5.57 | 9.63 | Both turns exact match |

| toolcall\_only | 2 | 1.000\* | 9.54 | 9.31 | 9.76 | Gating passed; schema wrapper issue — see note |

| mixed\_brief\_json | 2 | 1.000 | 10.59 | 10.39 | 10.79 | Answer line + JSON schema correct |

| toolcall | 2 | 1.000 | 14.39 | 14.25 | 14.53 | Both cases bucket=11 — clean at F16 |

| json | 4 | n/a | 64.02 | 33.87 | 149.88 | bucket\_score avg = 10.000; json\_01 outlier |

| fuzz | 20 | n/a | 26.61 | 10.19 | 57.78 | bucket\_score avg = 10.000 |

| mcq | 5 | n/a | 0.46 | 0.45 | 0.48 | bucket\_score avg = 0.600 — 2 failures |

json\_multistep — Case-Level Breakdown

|---|---|---|---:|---|

F16 passes both medium cases. At Q4\_K\_M, ms\_med\_02 additionally fails on checks_consistent_ok, and ms\_hard\_01 degrades from a partial failure (cc+oracle) to a complete failure across all four gating signals. This is the direct capability argument for F16 in planning-adjacent workloads.

The json_multistep failures at F16 are scoped: ms\_easy\_02 fails only on oracle equivalence (the model plans incorrectly but is internally consistent), and ms\_hard\_01 fails on both consistency and oracle (a harder failure that tracks with the difficulty level). No schema or STOP semantics failures occur at F16.

toolcall — Fully Clean at F16

Both tool_01 and tool_02 pass at bucket=11 — the maximum score. Tool parse is valid, schema is valid, and the final answer matches expected output. This contrasts with the Q4\_K\_M variant where tool_02 produces a final_mismatch (bucket=0) despite a valid tool dispatch.

|---|---:|---|---:|---|

| tool\_01 | 11 | ok | 11 | ok |

| tool\_02 | 11 | ok | 0 | final\_mismatch |

At F16, tool dispatch and post-tool answer accuracy are both reliable. If your application depends on the model correctly processing tool outputs and reporting accurate results — not just calling the right tool — F16 is the safer choice.

⚠️ toolcall\_only — Schema Wrapper Non-Compliance (F16)

toolcall_only passes gating at 1.000 (tool\_name\_ok=1, args\_ok=1) but carries schema_ok=0 on both cases with detail=schema_error. The model correctly identifies the tool and extracts valid arguments but wraps the output using "tool" as the outer key instead of the expected "tool_name".

| Signal | Rate | Tier |

|---|---:|---|

| tool\_name\_ok | 1.000 | Tier-1 (gating) ✅ |

| args\_ok | 1.000 | Tier-1 (gating) ✅ |

| schema\_ok | 0.000 | Non-gating (tracked) |

This is a schema discipline issue, not a capability failure. A one-line normalization step ("tool" → "tool_name") in the response parser resolves it for strict schema enforcement environments. See the Q4\_K\_M companion card for the normalize_tool_wrapper() implementation pattern.

MCQ — A-Bias at F16

mcq_02 and mcq_05 both fail with wrong_choice got=A. Both failures occur at 0.46 seconds — fast, confident, and wrong. This model defaults to option A when uncertain. The Q4\_K\_M variant adds mcq_04 to the failure set (also got=A), but the A-bias is a model-level characteristic present at full precision, not a quantization artifact.

| Case | Result | Detail |

|---|---|---|

| mcq\_01 | ✅ PASS | ok |

| mcq\_02 | ❌ FAIL | wrong\_choice got=A |

| mcq\_03 | ✅ PASS | ok |

| mcq\_04 | ✅ PASS | ok |

| mcq\_05 | ❌ FAIL | wrong\_choice got=A |

Signal-Level Diagnostics (F16)

json\_multistep

| Signal | Rate | Tier |

|---|---:|---|

| schema\_ok | 1.000 | Tier-1 (gating) |

| checks\_consistent\_ok | 0.800 | Tier-1 (gating) |

| stop\_semantics\_ok | 1.000 | Tier-1 (gating) |

| oracle\_equiv\_ok | 0.600 | Tier-1 (gating) |

| final\_consistent\_ok | 0.000 | Tier-2 (tracked, non-gating) |

| final\_match\_reported | 0.000 | Tier-2 (tracked, non-gating) |

schema_ok=1.000 and stop_semantics_ok=1.000 at F16 — both drop to 0.800 at Q4\_K\_M. These signal-level regressions are what drive the pass rate from 0.600 to 0.400 under quantization.

stateful\_followup

| Signal | Rate |

|---|---:|

| turn1\_parse\_ok | 1.000 |

| turn2\_parse\_ok | 1.000 |

| turn1\_exact\_match | 1.000 |

| turn2\_exact\_match | 1.000 |

toolcall\_only

| Signal | Rate | Tier |

|---|---:|---|

| tool\_name\_ok | 1.000 | Tier-1 (gating) |

| args\_ok | 1.000 | Tier-1 (gating) |

| schema\_ok | 0.000 | Non-gating (tracked) |

mixed\_brief\_json

| Signal | Rate |

|---|---:|

| answer\_line\_ok | 1.000 |

| json\_parse\_ok | 1.000 |

| schema\_ok | 1.000 |

When to Deploy F16 vs. Q4\_K\_M

| Criterion | F16 (this repo) | Q4\_K\_M |

|---|---|---|

| VRAM available | ~26 GB | ~10 GB |

| Latency acceptable | ~30 sec/case avg | ~1.4 sec/case avg |

| json\_multistep pass rate | 0.600 | 0.400 |

| ms\_med\_02 | ✅ PASS | ❌ FAIL |

| ms\_hard\_01 | ❌ partial fail (cc+oracle) | ❌ complete fail (all 4) |

| toolcall accuracy | ✅ Clean (both bucket=11) | ⚠️ tool\_02 final\_mismatch |

| toolcall\_only gating | ✅ 1.000 | ❌ 0.000 (args fail) |

| Schema wrapper norm | Needed | Not applicable |

| MCQ failures | 2/5 (mcq\_02, mcq\_05) | 3/5 (+mcq\_04) |

| 128K context at full precision | ✅ | Limited by VRAM |

Recommended Use Cases — F16

✅ Deploy with Confidence

Tool-calling pipelines requiring answer accuracy — Both toolcall cases pass at bucket=11 with no final\_mismatch. F16 is the correct choice when tool execution results feed downstream computation that must be correct.
Stateful multi-turn agents — Perfect two-turn state retention (1.000) at 7.60 sec/case avg.
Structured JSON outputs (single-step) — json and fuzz both achieve bucket\_score 10.000.
Hybrid brief + JSON responses — mixed_brief_json passes at 1.000.
Medium-difficulty multi-step planning — ms\_med\_01 and ms\_med\_02 both pass. F16 retains ms\_med\_02 that Q4\_K\_M loses.
Long-document processing at full precision — 128K context with F16 weights provides maximum fidelity for large-scale document Q&A, multi-document comparison, and long-form extraction.
Multilingual structured tasks — All 9 supported languages at maximum precision.
Tool-only dispatch with schema normalization — toolcall_only passes gating at 1.000 with a one-line wrapper key fix.

⚠️ Use with Guardrails

Hard multi-step planning — ms\_hard\_01 fails at F16 too (cc+oracle), though less catastrophically than at Q4\_K\_M. Use with an external validator for hard planning tasks.
MCQ with A-bias mitigation — Two of five cases fail at F16 with wrong\_choice got=A. Add chain-of-thought prompting or response validation for MCQ-style pipelines.

❌ Not Recommended

High-throughput pipelines — At 30.24 sec/case average and up to 149.88 seconds on a single json case, F16 is not suitable for latency-sensitive or batch workloads.

Hardware Requirements

| Configuration | VRAM Required | Recommended GPU |

|---|---|---|

| F16 (this repo) · full GPU offload | ~26 GB | A100 40 GB · 2× A10G · RTX 4090 (partial) |

| F16 · mixed CPU/GPU offload | 16–24 GB VRAM + 16 GB RAM | RTX 3090/4090 with n_gpu_layers tuning |

| Q4\_K\_M (companion repo) · 8K context | ~10 GB | T4 16 GB · RTX 3080 |

| Q4\_K\_M (companion repo) · 128K context | ~18 GB | A10G 24 GB · RTX 4090 |

Usage

Installation

pip install llama-cpp-python huggingface_hub

For GPU acceleration (CUDA):

CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python --force-reinstall --no-cache-dir

Python — llama-cpp-python

from huggingface_hub import hf_hub_download
from llama_cpp import Llama

# Note: 24.5 GB download — ensure sufficient disk space and ~26 GB VRAM
model_path = hf_hub_download(
    repo_id="pbhappliedsystems/mistral-nemo-instruct-2407-gguf-F16",
    filename="mistral-nemo-instruct-2407-gguf-F16.gguf"
)

llm = Llama(
    model_path=model_path,
    n_ctx=32768,      # Adjust to use case; supports up to 128K
    n_gpu_layers=-1,  # -1 offloads all layers to GPU; reduce if VRAM < 26 GB
    verbose=False,
)

response = llm.create_chat_completion(
    messages=[
        {
            "role": "system",
            "content": "You are a precise assistant. Follow instructions exactly and return structured outputs when requested."
        },
        {
            "role": "user",
            "content": "Analyze the following document and return a JSON object with keys: summary, key_entities, sentiment, action_items."
        }
    ],
    temperature=0.3,
    max_tokens=1024,
)

print(response["choices"][0]["message"]["content"])

For partial GPU offload when VRAM is between 16–24 GB:

llm = Llama(
    model_path=model_path,
    n_ctx=16384,
    n_gpu_layers=20,   # Tune based on available VRAM
    verbose=True,      # Enable to monitor layer offload and memory usage
)

For tool-calling with schema normalization (addresses the toolcall_only wrapper issue):

import json, re
from huggingface_hub import hf_hub_download
from llama_cpp import Llama

model_path = hf_hub_download(
    repo_id="pbhappliedsystems/mistral-nemo-instruct-2407-gguf-F16",
    filename="mistral-nemo-instruct-2407-gguf-F16.gguf"
)

llm = Llama(model_path=model_path, n_ctx=4096, n_gpu_layers=-1, verbose=False)

def normalize_tool_wrapper(raw: str) -> dict:
    """
    Normalize F16 schema wrapper non-compliance.
    Maps non-standard 'tool' key -> 'tool_name' before validation.
    See quant_eval v7.21 toolcall_only: schema_ok=0 at F16 (gating passes).
    """
    match = re.search(r'```(?:json)?\s*([\s\S]*?)```', raw)
    payload = match.group(1).strip() if match else raw.strip()
    parsed = json.loads(payload)
    if "tool" in parsed and "tool_name" not in parsed:
        parsed["tool_name"] = parsed.pop("tool")
    assert "tool_name" in parsed and "args" in parsed
    return parsed

response = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "Respond only with a valid JSON tool call."},
        {"role": "user", "content": "Add 5 and 10."}
    ],
    temperature=0.0,
    max_tokens=256,
)
result = normalize_tool_wrapper(response["choices"][0]["message"]["content"])
print(result)

CLI — llama-cli

llama-cli \
  --model mistral-nemo-instruct-2407-gguf-F16.gguf \
  --chat-template mistral \
  --system-prompt "You are a precise assistant." \
  --prompt "Analyze the following and return a JSON object with keys: summary, risk_level, action_items." \
  --n-predict 1024 \
  --ctx-size 32768 \
  --n-gpu-layers -1 \
  --temp 0.3

For server deployment:

llama-server \
  --model mistral-nemo-instruct-2407-gguf-F16.gguf \
  --chat-template mistral \
  --ctx-size 32768 \
  --n-gpu-layers -1 \
  --port 8080 \
  --host 0.0.0.0

Query via the OpenAI-compatible API:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-required")

response = client.chat.completions.create(
    model="mistral-nemo-instruct-2407-gguf-F16",
    messages=[{"role": "user", "content": "Your prompt here"}],
    temperature=0.3,
    timeout=180,  # Allow up to 3 minutes for long-context or complex json cases
)
print(response.choices[0].message.content)

Artifact Provenance

|---|---|---|---|

The F16 GGUF was converted from the mistralai/Mistral-Nemo-Instruct-2407 HuggingFace snapshot using a custom-built llama.cpp conversion pipeline developed by PBH Applied Systems, without modification to model weights.

Two-pass evaluation architecture: The F16 evaluation run (20260211_014604) operated in cache-generation mode (skip_quant=true), producing the full_weight_cache.json used as the reference baseline for the Q4\_K\_M comparison run (20260211_022944). This ensures that F16 and Q4\_K\_M results are measured against the identical fixture set under controlled, reproducible conditions.

Evaluation Methodology

quant_eval v7.21 is a proprietary behavioral evaluation harness developed by PBH Applied Systems. The two-run architecture evaluates the full-precision (F16) model first, caches its results, then evaluates the quantized variant against the same fixture set.

Fixture set: golden_oracle_fixtures_v7_21 (SHA256: 6d71a0b9147c079371b02a94f3c149eb78a6adc03dc16ff6833b964fbf4174f0)

| Family | Description | Pass Signals |

|---|---|---|

| fuzz | Property-based regression; structured placement correctness | schema\_ok, constraints\_ok |

| json | Single-step structured JSON with constraint rules | schema\_ok, constraints\_ok |

| json_multistep | Multi-step planning with self-check and oracle verification | schema\_ok, checks\_consistent\_ok, stop\_semantics\_ok, oracle\_equiv\_ok |

| mcq | Multiple-choice extraction | choice\_ok |

| stateful_followup | Two-turn state tracking; turn-2 correct given turn-1 | turn1/2\_parse\_ok, turn1/2\_exact\_match |

| mixed_brief_json | Hybrid: natural language answer + valid JSON block | answer\_line\_ok, json\_parse\_ok, schema\_ok |

| toolcall | Tool call embedded in response; parse + schema validation | stage1\_tool\_parse\_ok, stage1\_tool\_schema\_ok |

| toolcall_only | Bare schema-only tool call; strict tool name + args check | tool\_name\_ok, args\_ok |

Evaluation hardware: NVIDIA RTX 4090 (24 GB VRAM)

F16 evaluation date: February 11, 2026

quant_eval seed: 42

🔬 About quant_eval & This Evaluation Series

quant_eval is a proprietary behavioral evaluation harness developed by PBH Applied Systems, LLC. It measures real agent-adjacent task performance across structured output, tool dispatch, multi-turn state retention, and multi-step planning — not perplexity or leaderboard proxies. Every model published under pbhappliedsystems has been independently evaluated using quant_eval before being recommended for any production role.

See it in action: Live AI Agent Demo →

The demo runs production-style agent workflows powered by open-weight models selected through the quant_eval evaluation pipeline.

Need a deployment recommendation?

Not sure which quantization level is right for your hardware, latency target, or agent type?

→ pbhappliedsystems.com

Evaluated and published by PBH Applied Systems, LLC · patrick@pbhappliedsystems.com

About PBH Applied Systems

PBH Applied Systems, LLC is an Oklahoma City–based applied machine learning and AI systems company specializing in production-grade model evaluation, quantization pipelines, agentic AI infrastructure, and scalable AI-driven application development. The organization operates with a strong emphasis on engineering rigor, reproducibility, and real-world deployment constraints — particularly in environments where performance, cost efficiency, and reliability must be balanced against available hardware and budget.

Founder — Patrick Hill, M.S.

PBH Applied Systems was founded by Patrick Hill, a Data Scientist and AI/ML Engineer with 10+ years of experience delivering advanced analytics, predictive modeling, and decision-support solutions across high-stakes operational environments. Patrick holds a Master of Science in Software Engineering with concentrations in Artificial Intelligence and Machine Learning and a B.S. in Business Finance.

Technical expertise spans:

Languages & Data: Python, SQL, Linux, Pandas, NumPy, scikit-learn
ML & Modeling: Supervised and unsupervised learning, neural networks, NLP, transformers, regression, classification, forecasting, and feature engineering
AI/ML Frameworks: PyTorch, TensorFlow/Keras, HuggingFace Transformers, GGUF, llama.cpp, BitsAndBytes, PEFT, QLoRA
Deployment & MLOps: Flask APIs, Docker, CI/CD pipelines, REST endpoints, streaming inference, version control
Data Platforms: Jupyter, Databricks, Power BI, Matplotlib
Quantization: GGUF conversion, Q4\_K\_M / Q5\_K\_M / Q8\_0 strategies, adapter-per-model evaluation architecture

Published Author

Patrick is the author of Applied Machine Learning: Concepts, Tools, and Case Studies — a 1,200+ page practitioner-oriented textbook adopted as required reading for CSC 373 – Machine Learning at the University of Advancing Technology.

Core Service Areas

1. LLM Optimization & Deployment — End-to-end GGUF conversion and quantization with custom llama.cpp pipelines and adapter-per-model architecture.

2. AI Evaluation Frameworks — Proprietary behavioral evaluation via quant_eval: per-family pass rates, F16 vs. quantized delta analysis, failure cluster diagnostics, deployment recommendations.

3. Agentic AI Infrastructure — LlamaIndex ReAct agents, Flask orchestration, serverless GPU inference, full pipeline from model selection to production serving.

4. Scalable AI Application Development — Multimodal applications (quantized LLMs + Whisper + BLIP), Dockerized Flask APIs, advanced time-series forecasting with custom attention mechanisms, Bayesian hyperparameter optimization, and FinBERT sentiment fusion.

5. ML Pipeline Design & Analytics — Feature engineering, forward-chaining cross-validation, KPI dashboards, analytical governance at scale.

6. Model & Agent Cataloging — Structured catalog publishing with reproducible artifacts and clear performance tradeoff documentation.

📞 Work With PBH Applied Systems

This F16 card documents clean tool execution that degrades at Q4\_K\_M, medium planning cases that survive F16 but fail under quantization, and an MCQ A-bias that is a model characteristic — not a precision artifact. The Q4\_K\_M companion card maps exactly where each of these findings changes under quantization. The decision between F16 and Q4\_K\_M should be made with this data, not guessed at.

👉 Book a Scoping Call — Discuss your model selection, quantization strategy, or deployment architecture directly with Patrick.

👉 Request an Evaluation Report — A full quant_eval behavioral audit: per-family pass rates, F16 vs. quantized delta analysis, failure cluster diagnostics, and a deployment recommendation. Engagements from $2,500.

Connect

| | |

|---|---|

| 🌐 Website | pbhappliedsystems.com |

| 📧 Email | patrick@pbhappliedsystems.com |

| 💼 LinkedIn | PBH Applied Systems, LLC |

| ▶️ YouTube | @pbhappliedsystems |

| 📸 Instagram | @pbhappliedsystems |

| 👍 Facebook | pbhappliedsystems |

License

This GGUF repository inherits the license of the base model:

Apache 2.0 — mistralai/Mistral-Nemo-Instruct-2407

The quant_eval evaluation methodology, fixture set, and scoring framework are proprietary to PBH Applied Systems, LLC and are not included in this repository.

GGUF conversion and behavioral evaluation performed by PBH Applied Systems, LLC · quant_eval v7.21 · F16 Run ID: 20260211_014604

pbhappliedsystems/mistral-nemo-instruct-2407-gguf-F16

Mistral-Nemo-Instruct-2407 · GGUF F16

Try the Live AI Agent Demo

Model Description

Key Characteristics

PBH Applied Systems Evaluation — quant\_eval v7.21

Per-Family Pass Rates — F16 (full_weight_transformers)

json\_multistep — Case-Level Breakdown

toolcall — Fully Clean at F16

⚠️ toolcall\_only — Schema Wrapper Non-Compliance (F16)

MCQ — A-Bias at F16

Signal-Level Diagnostics (F16)

json\_multistep

stateful\_followup

toolcall\_only

mixed\_brief\_json

When to Deploy F16 vs. Q4\_K\_M

Recommended Use Cases — F16

✅ Deploy with Confidence

⚠️ Use with Guardrails

❌ Not Recommended

Hardware Requirements

Usage

Installation

Python — llama-cpp-python

CLI — llama-cli

Artifact Provenance

Evaluation Methodology

🔬 About quant_eval & This Evaluation Series

About PBH Applied Systems

Founder — Patrick Hill, M.S.

Published Author

Core Service Areas

📞 Work With PBH Applied Systems

Connect

License

Per-Family Pass Rates — F16 (`full_weight_transformers`)