Qwen2.5-7B-Instruct · GGUF F16

Converted and evaluated by PBH Applied Systems, LLC

— Applied AI/ML Consulting · LLM Optimization & Deployment · Quantized AI Infrastructure

🔬 This repository is part of a production-oriented evaluation series. Every model published under pbhappliedsystems has been independently evaluated using quant_eval v7.21 — a proprietary behavioral evaluation harness developed by PBH Applied Systems.

📌 This is the full-precision F16 baseline repository. The evaluated Q4\_K\_M deployment variant is published at pbhappliedsystems/qwen-2.5-7B-instruct-gguf-Q4-K-M. That card documents the complete comparison including stateful recovery (0.000 → 1.000), mixed\_brief\_json recovery (0.500 → 1.000), and the F16-specific hallucination patterns documented below.

Try the Live AI Agent Demo

Launch the PBH Applied Systems AI Agent Demo →

This model is part of the PBH Applied Systems evaluated model series that supports the live AI Agent Demo. The demo lets visitors interact with production-style agent workflows powered by open-weight language models evaluated through PBH Applied Systems' quant_eval framework.

The F16 model serves a different role than the Q4_K_M deployment variant. F16 is the full-precision baseline used to measure what the model can do before quantization. quant_eval then compares the quantized model against this baseline to identify which capabilities are preserved, which degrade, and which tasks require guardrails or a higher-precision deployment.

This comparison is central to the demo. It helps determine which model belongs in which agent role:

Reasoning models are selected for planning, analysis, and auditable decision workflows.
Document models are selected for long-context extraction, summarization, and structured Q&A.
Code models are selected for task completion, structured output, API scaffolding, and automation workflows.
Quantized variants are selected when they preserve enough behavior to reduce cost, latency, and GPU requirements.
F16 variants remain important when maximum fidelity, cleaner tool execution, or reduced quantization risk matters more than speed or cost.

The live demo shows the deployment side of that process. The F16 card documents the reference behavior. The Q4_K_M card shows what changes after compression. Together, they explain how PBH Applied Systems uses quant_eval to choose the correct LLM for the correct agent type instead of guessing from model size or leaderboard reputation.

Model Description

This repository contains the full-precision F16 GGUF of Qwen/Qwen2.5-7B-Instruct, a 7-billion parameter instruction-tuned model from Alibaba Cloud.

In the PBH Applied Systems evaluation pipeline, this F16 run (20260221_024142) operated in cache-generation mode (skip_quant=true), producing the full_weight_cache.json used as the reference baseline for the Q4\_K\_M comparison run (20260221_024911). The F16 evaluation data here is confirmed to match the F16 baseline in the comparison run — identical timing profiles and raw output content validate clean cache reuse.

Key Characteristics

Parameters: 7B
Format: GGUF F16 (full precision)
File size: 15.2 GB
SHA256: ebb1cb9f8d7721f5ea509ff9e7327873b039e523c17f2d03d6f1b90729574b54
Minimum VRAM (GPU inference): ~18 GB
Recommended GPU tier: A10G 24 GB · RTX 4090 · 2× RTX 3080
Context window: 32,768 tokens
Inference speed (eval hardware): avg 1.688 sec/case on RTX 4090
License: Qwen Research License (non-commercial)

PBH Applied Systems Evaluation — quant\_eval v7.21

Evaluation conducted by PBH Applied Systems, LLC using quant_eval v7.21

Run ID: 20260221_024142 · Fixtures: golden_oracle_fixtures_v7_21 (SHA256: 6d71a0b9147c...) · Seed: 42

Hardware: NVIDIA RTX 4090 · Runner: full_weight_transformers (F16 only) · Total rows: 42

Per-Family Pass Rates — F16 (`full_weight_transformers`)

|---|---:|---:|---:|---:|---|

| json\_multistep | 5 | 0.800 | 3.812 | 2.200 | ms\_easy\_02 only failure |

| stateful\_followup | 2 | 0.000 | 0.760 | 1.000 | Turn-2 PARTICULAR: hallucination |

| toolcall\_only | 2 | 0.000 | 1.300 | 0.500 | PHP code on toolonly\_02 |

| mixed\_brief\_json | 2 | 0.500 | 0.530 | 1.500 | mixed\_02 garbled swer: ANSWER: prefix |

| toolcall | 2 | 1.000 | 0.905 | 0.000 | Stage-1 passes; role-token final\_mismatch |

| json | 4 | n/a | 2.610 | 10.000 | All pass |

| fuzz | 20 | n/a | 1.713 | 10.000 | All 20 pass |

| mcq | 5 | n/a | 0.030 | 0.000 | Empty output on all 5 |

F16-Specific Findings

All findings below are documented in full in the Q4\_K\_M companion card. This section provides the F16-specific evidence with raw output detail.

Finding 1: json\_multistep — Single Precision-Invariant Failure

|---|---|---:|---|

| ms\_easy\_01 | ✅ PASS | 2.635 | — |

| ms\_med\_01 | ✅ PASS | 4.547 | — |

| ms\_med\_02 | ✅ PASS | 4.151 | — |

| ms\_hard\_01 | ✅ PASS | 3.983 | — |

The F16 model passes the hard case (3.983s) and both medium cases. The sole failure — ms\_easy\_02 — occurs at identical signals on both F16 and Q4\_K\_M: oracle_equiv_ok=0, checks_consistent_ok=1. The model produces a plan that is internally self-consistent but arrives at the wrong final placement. This is a model behavior on this specific fixture, not a precision effect.

json\_multistep signal rates at F16 are perfect except oracle\_equiv\_ok:

| Signal | F16 Rate | Q4\_K\_M Rate |

|---|---:|---:|

| schema\_ok | 1.000 | 1.000 |

| checks\_consistent\_ok | 1.000 | 1.000 |

| stop\_semantics\_ok | 1.000 | 1.000 |

| oracle\_equiv\_ok | 0.800 | 0.800 |

Finding 2: stateful\_followup — Turn-2 `PARTICULAR:` Hallucination

Both stateful cases fail on turn-2 with the HuggingFace Transformers runner appending an unsolicited annotation after the correct JSON state:

| Case | Turn 1 | Turn 2 Raw |

|---|---|---|

| state\_01 | ✅ {"counter": 2} | ❌ user {"counter": 2} PARTICULAR: The assistant will add 3 to the previous counter value a... |

| state\_02 | ✅ {"items": ["a", "b"]} | ❌ user `json {"items": ["a", "b"]} ` PARTICULAR: The assistant should append "c" to th... |

Turn 1 is correct on both cases (turn1_exact_match=1). Turn 2 correctly produces the state JSON but then continues generating a PARTICULAR: explanatory annotation that corrupts extraction. The Q4\_K\_M runner produces clean JSON state with EOS tokens and passes at 1.000. This is a runner behavior, not a model capability failure.

Finding 3: toolcall\_only — PHP Hallucination on toolonly\_02

toolonly_02 at F16 produces PHP code instead of JSON:

').\" + $result;
         echo json_encode(array("tool_calls" => array($tool_call), "answer"

toolonly_01 produces Assistant {"tool": "add", "a": 5, "b": 10} — wrong schema but valid JSON. The Q4\_K\_M runner produces {"tool": "add", "numbers": [5, 10]} on both cases — consistently wrong schema but consistently JSON. Both runners fail toolcall_only, but F16 toolonly\_02 is the more severe failure: no JSON at all vs. wrong-schema JSON.

Finding 4: toolcall — Correct Dispatch, Role-Token Final Mismatch

Both toolcall cases pass stage-1 (s1p=1, s1s=1) but produce final_mismatch:

| Case | Raw Output | Why |

|---|---|---|

| tool\_01 | user {...add(2,3)...} round | Role token "user" before JSON; "round" instead of "5" |

| tool\_02 | user {...add(10,-4)...} pageNumber: 6 | Role token before JSON; "pageNumber: 6" instead of "6" |

The tool dispatch JSON is valid. The final answer is present but wrapped in extraneous tokens: pageNumber: 6 (tool\_02) and round (tool\_01 — likely a truncated word). The Q4\_K\_M runner produces 5<|im_end|> and 6<|im_end|> — correct arithmetic with EOS contamination. Both are final_mismatch but the F16 answers contain non-numeric garbage while Q4\_K\_M answers contain correct numbers with strippable tokens.

Finding 5: mixed\_brief\_json — Garbled ANSWER Prefix on mixed\_02

| Case | Raw | Result |

|---|---|---|

| mixed\_01 | ANSWER: 13 {"a":4,"b":9,"sum":13} | ✅ PASS (leading space stripped) |

| mixed\_02 | swer: ANSWER: 6 {"a": -2, "b": 8, "sum": 6} | ❌ FAIL (answer_line_ok=0) |

mixed\_01 passes despite a leading space because the extractor finds ANSWER: correctly. mixed\_02 fails because the model partially generates the word "answer:" then writes ANSWER: again, producing swer: ANSWER: — a doubled prefix that the extractor cannot parse. The JSON block is valid (json_parse_ok=1, schema_ok=1). The Q4\_K\_M runner produces clean ANSWER: 6 {...} on both cases.

F16 vs. Q4\_K\_M — Deployment Decision

| Family | F16 | Q4\_K\_M | Verdict |

|---|---|---|---|

| json\_multistep | 0.800 | 0.800 | Equal — same single failure |

| stateful\_followup | 0.000 | 1.000 | Q4\_K\_M wins — runner handles turn-2 cleanly |

| toolcall\_only | 0.000 | 0.000 | Equal — both fail args schema |

| mixed\_brief\_json | 0.500 | 1.000 | Q4\_K\_M wins — runner handles ANSWER prefix cleanly |

| toolcall (stage-1) | 1.000 | 1.000 | Equal |

| json / fuzz | 10.000 / 10.000 | 10.000 / 10.000 | Equal |

| VRAM required | ~18 GB | ~6 GB | Q4\_K\_M wins |

For this model, Q4\_K\_M is the stronger deployment choice across every operational dimension. The F16 runner exhibits hallucination patterns (PARTICULAR annotation, PHP code, garbled prefixes) that the Q4\_K\_M llama.cpp runner avoids. The only scenario where F16 is preferred is when ~18 GB VRAM is available and maximum context is needed with full-weight fidelity.

Signal-Level Diagnostics (F16)

json\_multistep

| Signal | Rate | Notes |

|---|---:|---|

| schema\_ok | 1.000 | Perfect |

| checks\_consistent\_ok | 1.000 | Perfect — 7B resolves 3B ceiling |

| stop\_semantics\_ok | 1.000 | Perfect |

| oracle\_equiv\_ok | 0.800 | ms\_easy\_02 only — precision-invariant |

stateful\_followup

| Signal | Rate |

|---|---:|

| turn1\_parse\_ok | 1.000 |

| turn2\_parse\_ok | 0.000 |

| turn1\_exact\_match | 1.000 |

| turn2\_exact\_match | 0.000 |

toolcall\_only

| Signal | Rate |

|---|---:|

| tool\_name\_ok | 0.500 |

| args\_ok | 0.000 |

mixed\_brief\_json

| Signal | Rate |

|---|---:|

| answer\_line\_ok | 0.500 |

| json\_parse\_ok | 1.000 |

| schema\_ok | 1.000 |

Hardware Requirements

| Configuration | VRAM Required | Notes |

|---|---|---|

| F16 (this repo) · full GPU | ~18 GB | 15.2 GB model + KV cache |

| F16 · partial CPU offload | 12–16 GB VRAM + 8 GB RAM | RTX 3080/3090 with n_gpu_layers tuning |

| Q4\_K\_M (companion repo) | ~6 GB | 4.68 GB — T4 16 GB · RTX 3060 |

Usage

Installation

pip install llama-cpp-python huggingface_hub

For GPU acceleration (CUDA):

CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python --force-reinstall --no-cache-dir

Python — llama-cpp-python (recommended for F16)

from huggingface_hub import hf_hub_download
from llama_cpp import Llama

# Note: 15.2 GB download — ensure sufficient disk space and ~18 GB VRAM
model_path = hf_hub_download(
    repo_id="pbhappliedsystems/qwen-2.5-7B-instruct-gguf-F16",
    filename="qwen-2.5-7B-instruct-gguf-F16.gguf"
)

llm = Llama(
    model_path=model_path,
    n_ctx=8192,
    n_gpu_layers=-1,
    verbose=False,
)

response = llm.create_chat_completion(
    messages=[
        {
            "role": "system",
            "content": "You are a precise assistant. Follow instructions exactly and return structured outputs when requested."
        },
        {
            "role": "user",
            "content": "Analyze the following and return a JSON object with keys: summary, risk_level, action_items."
        }
    ],
    temperature=0.15,
    max_tokens=1024,
)
print(response["choices"][0]["message"]["content"])

For stateful multi-turn use — strip the PARTICULAR: annotation if it appears (F16 runner behavior):

import json, re

def extract_clean_state(raw: str) -> dict:
    """
    Strip F16 role tokens and PARTICULAR: annotations from stateful responses.
    quant_eval v7.21 finding: turn2 appends 'PARTICULAR: The assistant will...'
    after the correct JSON state, blocking extraction.
    """
    # Remove role-token prefixes
    clean = re.sub(r'^(user|assistant|ician|ed user)\s*', '', raw.strip())
    # Strip markdown code fences
    clean = re.sub(r'```json\s*|```', '', clean).strip()
    # Extract only up to the first JSON object
    match = re.match(r'(\{[^}]+\})', clean)
    if match:
        return json.loads(match.group(1))
    raise ValueError(f"No JSON state found in: {raw[:100]}")

response = llm.create_chat_completion(messages=conversation, temperature=0.15, max_tokens=256)
state = extract_clean_state(response["choices"][0]["message"]["content"])

For partial GPU offload when VRAM is 12–16 GB:

llm = Llama(
    model_path=model_path,
    n_ctx=4096,
    n_gpu_layers=24,   # Tune based on available VRAM
    verbose=True,
)

CLI — llama-cli

llama-cli \
  --model qwen-2.5-7B-instruct-gguf-F16.gguf \
  --chat-template qwen2 \
  --system-prompt "You are a precise assistant. Follow instructions exactly." \
  --prompt "Return a JSON object with keys: summary, risk_level, action_items." \
  --n-predict 1024 \
  --ctx-size 8192 \
  --n-gpu-layers -1 \
  --temp 0.15

For server deployment:

llama-server \
  --model qwen-2.5-7B-instruct-gguf-F16.gguf \
  --chat-template qwen2 \
  --ctx-size 8192 \
  --n-gpu-layers -1 \
  --port 8080 \
  --host 0.0.0.0

Query via the OpenAI-compatible API:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-required")

response = client.chat.completions.create(
    model="qwen-2.5-7B-instruct-gguf-F16",
    messages=[{"role": "user", "content": "Your prompt here"}],
    temperature=0.15,
)
print(response.choices[0].message.content)

Artifact Provenance

|---|---|---|---|

The F16 GGUF was converted from Qwen/Qwen2.5-7B-Instruct using a custom-built llama.cpp conversion pipeline developed by PBH Applied Systems.

Two-pass architecture: This F16 run (20260221_024142) operated in cache-generation mode (skip_quant=true). Timing identity between this run and the F16 baseline in the Q4_K_M comparison run confirms clean cache reuse and run integrity.

Evaluation Methodology

quant_eval v7.21 — proprietary behavioral evaluation harness, PBH Applied Systems.

Fixture set: golden_oracle_fixtures_v7_21 (SHA256: 6d71a0b9147c079371b02a94f3c149eb78a6adc03dc16ff6833b964fbf4174f0)

Evaluation hardware: NVIDIA RTX 4090 · F16 evaluation date: February 21, 2026 · Seed: 42

🔬 About quant_eval & This Evaluation Series

quant_eval is a proprietary behavioral evaluation harness developed by PBH Applied Systems, LLC. It measures real agent-adjacent task performance across structured output, tool dispatch, multi-turn state retention, and multi-step planning — not perplexity or leaderboard proxies. Every model published under pbhappliedsystems has been independently evaluated using quant_eval before being recommended for any production role.

See it in action: Live AI Agent Demo →

The demo runs production-style agent workflows powered by open-weight models selected through the quant_eval evaluation pipeline.

Need a deployment recommendation?

Not sure which quantization level is right for your hardware, latency target, or agent type?

→ pbhappliedsystems.com

Evaluated and published by PBH Applied Systems, LLC · patrick@pbhappliedsystems.com

About PBH Applied Systems

PBH Applied Systems, LLC is an Oklahoma City–based applied machine learning and AI systems company specializing in production-grade model evaluation, quantization pipelines, agentic AI infrastructure, and scalable AI-driven application development.

Patrick Hill, M.S. — Founder · Data Scientist · AI/ML Engineer · Author of Applied Machine Learning: Concepts, Tools, and Case Studies (required reading, UAT CSC 373)

📞 Work With PBH Applied Systems

The F16 evaluation documents two things clearly: what the 7B model is capable of (planning that clears the 3B reasoning ceiling, perfect fuzz stability), and where the HuggingFace Transformers runner introduces artifacts that the llama.cpp Q4\_K\_M runner avoids. Both precision levels tell part of the story. Only together do they tell all of it.

👉 Book a Scoping Call · 👉 Request an Evaluation Report — from $2,500

Connect

| | |

|---|---|

| 🌐 | pbhappliedsystems.com |

| 📧 | patrick@pbhappliedsystems.com |

| 💼 | LinkedIn |

| ▶️ | YouTube |

| 📸 | Instagram |

| 👍 | Facebook |

License

This GGUF repository inherits the license of the base model:

Apache 2.0 — Qwen/Qwen2.5-7B-Instruct

The quant_eval evaluation methodology, fixture set, and scoring framework are proprietary to PBH Applied Systems, LLC and are not included in this repository.

GGUF conversion and behavioral evaluation performed by PBH Applied Systems, LLC · quant_eval v7.21 · F16 Run ID: 20260221_024142

pbhappliedsystems/qwen-2.5-7B-instruct-gguf-F16

Qwen2.5-7B-Instruct · GGUF F16

Try the Live AI Agent Demo

Model Description

Key Characteristics

PBH Applied Systems Evaluation — quant\_eval v7.21

Per-Family Pass Rates — F16 (full_weight_transformers)

F16-Specific Findings

Finding 1: json\_multistep — Single Precision-Invariant Failure

Finding 2: stateful\_followup — Turn-2 PARTICULAR: Hallucination

Finding 3: toolcall\_only — PHP Hallucination on toolonly\_02

Finding 4: toolcall — Correct Dispatch, Role-Token Final Mismatch

Finding 5: mixed\_brief\_json — Garbled ANSWER Prefix on mixed\_02

F16 vs. Q4\_K\_M — Deployment Decision

Signal-Level Diagnostics (F16)

json\_multistep

stateful\_followup

toolcall\_only

mixed\_brief\_json

Hardware Requirements

Usage

Installation

Python — llama-cpp-python (recommended for F16)

CLI — llama-cli

Artifact Provenance

Evaluation Methodology

🔬 About quant_eval & This Evaluation Series

About PBH Applied Systems

📞 Work With PBH Applied Systems

Connect

License

Per-Family Pass Rates — F16 (`full_weight_transformers`)

Finding 2: stateful\_followup — Turn-2 `PARTICULAR:` Hallucination