Qwen Agentic Chat Template for llama.cpp

A Jinja chat template for Qwen 3.5 / 3.6 / 3.8 tuned for llama.cpp / ik_llama driving agentic coding harnesses (OpenCode, Claude Code, Hermes, aider, and others).

Built on froggeric/Qwen-Fixed-Chat-Templates v21.3 with an agentic hardening layer on top: structural loop detection, tool-failure classification, vanished-tool detection, argument-grounding rules, and reasoning-effort steering. Releases are grounded in captured harness traffic and verified on the real llama.cpp renderer.

Qwen 3.8

v20 stages skills around the active work item and keeps long-running todo plans current without turning every tool result into a planning interruption. The base grammar is unchanged, so one file still covers 3.5, 3.6 and 3.8.

• Skills are action-staged, not globally capped. The model identifies the complete match set, then loads only the primary match for the immediate action and its explicit dependencies, at most three per response. Deferred matches load when their work item becomes active; a completed batch proceeds directly to a concrete non-skill action.

• Todo plans are reconciled in bounded windows. When a configured todo tool is available, every fourth completed non-todo/non-skill action batch opens a review checkpoint. Completed items are checked from evidence, partial items stay open, and unchanged plans remain silent. A todo transition can load a newly direct skill chain.

• Field evidence validates the final checkpoint behavior. In the final Qwen3.8-27B run, all three skill batches transitioned to useful non-skill actions with 614, 5,339, and 116 reasoning characters; todo completion advanced from one to eight without reconciliation spam.

• reasoning_effort has four distinct tiers: low, medium, high, and xhigh. high is no longer an alias for xhigh. Each tier preserves a different analysis depth while bounding investigation, whole-task rehearsal, and duplicate drafting before tool calls. An unrecognised value degrades to medium rather than raising, and all steering text is suppressed automatically when thinking is off.

• The default remains medium, now with explicit turn-local guidance. It reasons about the immediate unresolved decision, acts on safe reversible steps, validates a failed verifier before expanding, and changes strategy when two probes do not narrow the same issue. In the original controlled A/B on Qwen3.8-27B, the stock template under xhigh averaged 30,102 reasoning characters below 40k context and produced two output truncations, including the harness's own conversation summary.

• Recommended temperature: 0.70 for Qwen3.6 and Qwen3.8 agentic use with this template. This is this project's starting point, not an upstream default, and it is a deliberate midpoint. Qwen3.6's official guidance uses 1.0 for general thinking-mode tasks and 0.6 for precise coding tasks; agentic work is both. Earlier releases of this template recommended 0.60, which several users found too conservative for open-ended building. 0.70 keeps tool-call and path accuracy close to the low end while restoring some of the variety 1.0 provides. If your work is mostly precise edits to existing code, 0.60 remains reasonable; if it is mostly open-ended generation, try 0.80. The remaining sampler values are unchanged.

• Consecutive leading system / developer messages are merged into one system block. Reading only messages[0] silently discards the rest, which breaks clients that send both.

• preserve_reasoning is accepted as an alias for preserve_thinking, so llama.cpp's --reasoning-preserve CLI flag reaches the template. An explicit preserve_thinking still wins.

Install

llama-server ... --jinja --chat-template-file chat_template.jinja

Modern builds auto-derive the tool-call parser, and the default XML format works out of the box (large multiline writes included - the server escapes, not the model). Older parsers that only read Hermes JSON need --chat-template-kwargs '{"tool_call_format":"json"}' (symptom without it: tool calls appear as plain <function=...> text).

Set kwargs on the llama-server command line, not the client config - some harnesses silently drop client-side chat_template_kwargs (OpenCode #26233).

What it adds over upstream

Structural loop detection (template-level; samplers only catch verbatim token loops, not action loops). Three detectors, each covering what the others cannot:

• No-progress - a tool returns identical results repeat_nudge_after times (default 6). Skips mutating tools (mutating_tools kwarg): writing six files returns six identical "success" strings that are productive, not stuck. Measured on 381 real sessions, this fires ~90% less than naive counting while keeping every genuine catch (scripts/measure_loop_nudges.py). Includes a polling escape hatch: deliberate wait-and-check patterns can declare themselves.
• Action loop / ping-pong - the model repeats the same call (tool + arguments) while the results differ every time (screenshots, timestamps, or harness message IDs make results unique and blind the no-progress detector). Call signatures are compared against the previous two turns, catching A→A→A and A→B→A→B patterns. pingpong_nudge_after (default 4). Deliberately does not skip mutating tools - re-issuing the same write with the same arguments is a loop. The warning includes a polling escape hatch so deliberate wait-and-check patterns can declare themselves. If the same action/result pattern continues for two more turns after the checkpoint, a second-stage warning requires a materially different action, completion from existing evidence, or a concise blocker report.
• Consecutive errors - failure loops, with escalating warnings. OpenCode StatusCode: non 2xx status code ... results are classified as harness failures alongside MCP and script-execution errors.

Vanished-tools note. If a tool was called earlier in the session but is missing from the current tools[], its server likely died mid-session (observed in the field: a model killed its own MCP servers with a broad process kill, then improvised around the gap for 40 turns without noticing). The template detects the mismatch structurally and injects a system note naming the missing tools. The note bans pretending the tools exist and bans repairing servers, but allows safe continuation with the remaining tools. warn_vanished_tools (default true) - set it to false if your harness legitimately changes the tool list per request (just-in-time tool loading; meta-tool patterns such as mcp_search are fine because the meta-tool stays listed).

Tool-failure classification that tells the truth. Output starting with Traceback/Error:/fatal: marks a failed tool - unless the tool is a read/query tool (read_tools kwarg): a read returning a log file that begins with Error: is a successful read of error-shaped content, not a tool failure. Harness-generated envelopes (invalid input for tool, json parsing failed, unknown tool, ...) count as failures for every tool class. Output length is evidence, not a classifier gate: the warning notes that short error-shaped output often indicates total failure while longer output may include partial success, then leaves the model to inspect state and decide. Deliberately never matches user permission rejections - scolding the model into retrying after a human said no is harmful. Warnings are worded to let the model self-classify: an expected failure mid-fix (a failing test being worked on) is told to continue its normal cycle, and consecutive-error warnings distinguish an identical repeated error (change the approach) from changing errors (normal burn-down - continue).

Argument grounding rules. The tool instructions require every required argument value to come from the user request, visible repository state, the tool schema, or a prior tool result - and require edit/write paths and replacement strings to have been observed before editing. This targets hallucinated file paths and invented replacement strings, the most expensive local-agent failure.

Skill triggering and opening decomposition (v14-v16). Harnesses like OpenCode publish a skill catalog and a skill tool; frontier models load matching skills as a matter of course, while local Qwen models often read the catalog and then ignore it. v14 made skill use structural with a one-time opening checkpoint and point-of-need clauses in loop warnings. v15 adds a compact checkpoint on every later genuine user request: it lists successfully loaded skills, keeps their guidance active, and directs the model to load only newly direct matches when the work changes. v16 extends the opening pass when an exact task or agent tool is also present: after the skill-only response completes and every selected skill result returns, the model names independently runnable work before planning and hands those parts to the detected delegation tool in parallel where appropriate. Non-research delegation remained flat, while visible planning increased. One cumulative-v16 field opening mixed a task call into the skill response, so the phase ordering is now explicit and remains under validation. Harness-owned tool-response and screenshot envelopes are excluded. Skill behavior remains gated on a tool named exactly skill; the decomposition clause additionally requires an exact task or agent tool. Weak or incidental skill matches are explicitly excluded. Prefer no template-driven skill triggering? Use archive/chat_template-v13.jinja.

v18 bounded delegation and continuation routing. v18 replaces v16's broad opening decomposition with at most one clearly independent result needed for the next action, a decision-sized brief, and a concrete stopping condition; an empty delegation set is valid. A newly encountered failure domain can also trigger a directly relevant skill during an uninterrupted tool loop instead of waiting for another user message.

Two-sided turn management. The tool instructions balance continuation against stopping: the model is told to keep working until the request is complete (verify each change, then move on) and to prefer small concrete actions over long deliberation, while a bounded set of exit conditions (task done, blocked, needs user input, or a meaningful checkpoint worth showing) tells it when to stop. This replaced an earlier one-sided instruction whose trigger - "if you have all necessary data" - was satisfied after every successful tool call, and it is deliberately tuned to avoid both premature mid-task stops and reasoning spirals.

Harness-aware error markers - adds json parsing failed, unknown tool, tool execution failed, Script execution failed:, MCP error ..., StatusCode: non 2xx status code ..., PowerShell null-expression failures, err:, and similar to upstream's CLI-oriented list.

Configuration

Everything is a kwarg:

Put all kwargs in a single JSON object - a second --chat-template-kwargs flag overrides, it does not merge. A fully loaded example:

--chat-template-kwargs '{"tool_call_format":"json","repeat_nudge_after":4,"think_on_tool_failure":true,"max_tool_response_chars":4000,"mutating_tools":"write_file patch terminal execute_code todo memory skill_manage read_file"}'

On Windows PowerShell the inner quotes need escaping: --chat-template-kwargs '{\"tool_call_format\":\"json\"}'.

Files

• chat_template.jinja / chat_template_oneline.txt - current-release install aliases
• chat_template_v20.1.jinja / chat_template_v20.1_oneline.txt - explicitly versioned v20.1 copies
• tests/test_template.py - render-test suite (run after any edit)
• scripts/measure_loop_nudges.py - reproduce the loop-nudge metric on your own opencode.db
• archive/ - every prior version plus a changelog table

Publishing

A local release commit and live llama.cpp deployment do not publish a release to the Hugging Face repository. Hub publication is a separate, explicitly authorized step.

Before publishing:

1. Select and record the immutable local release commit or tag. Materialize the publishing tree from that revision, not from a later experimental HEAD.
2. Run the full render suite, live llama.cpp/minja checks, and git diff --check.
3. Verify the generic aliases, explicit versioned copies, and archived prior release.
4. Upload only the publishable root files, archive/, scripts/, and tests/. Exclude workbench/, local configuration, logs, backups, and unrelated untracked files.
5. Use the Hugging Face hf upload workflow with an explicit commit message. Do not use a normal or force Git push when local and Hub histories have diverged, and do not infer publication state from a stale local tracking ref.
6. Verify the Hub commit/tree through the official API and round-trip download the changed artifacts to compare hashes.
7. Record both the immutable local source revision and resulting Hub commit ID in the release notes or handoff.

See the official Hugging Face upload guide.

llama.cpp notes

• Grammar enforcement is active on auto-parser builds (b8227+) - malformed tool calls are token-level impossible (verified to 163 tool schemas). It cannot stop max_tokens truncation, so keep your output limit above reasoning budget + largest tool call.
• String arguments are normalized by mainline llama.cpp. Harnesses like OpenCode send historical tool-call arguments as JSON strings, but current mainline parses them into objects in C++ (func_args_not_string, common/chat.cpp) before the template renders - so they arrive as mappings and render as clean <parameter=> blocks. On renderers without that normalization (older forks, direct Jinja), string arguments render inside a labeled <arguments_json> compatibility block instead of masquerading as XML syntax, and the instructions tell the model never to emit that block in new calls. v19 extends this to every other shape: arguments that are neither a mapping nor a string (a bare list or number) are serialized as JSON instead of being silently dropped, and scalar parameter values render as JSON literals (true / false / null) rather than Python reprs, matching the official Qwen templates.
• JSON mode drops grammar enforcement. The auto-parser only grammar-constrains the XML format (verified: tool_choice: required is not enforced in JSON mode). On a measured A/B with identical planning prompts, JSON mode also caused a 26-73× reasoning-length blowup on this model class - XML is the default for a reason.
• Media marker: the template strips <__media__> only from system/assistant/reasoning content (roles that cannot hold a real image); user/tool markers are left intact so vision and tool screenshots work. A literal marker in user/tool text (for example, pasting this template into chat) 400s the server and cannot be fixed in-template - on mainline llama.cpp set LLAMA_MEDIA_MARKER; ik_llama does not support it, so avoid pasting the raw token.

Design notes

Rationale that used to live in template comments (the template itself is now comment-free; the single-line variant always was):

• Media-marker guards. llama.cpp flattens images into string content as bitmap-backed marker tokens. The guards test for the marker's presence first and never touch normal content. They strip only from roles that cannot legitimately hold an image; stripping user/tool markers would desync markers vs. bitmaps and 400 the server (a v3/v7 unconditional strip silently broke real vision and was reverted in v8). Automated reviewers: HTML-interpreting fetchers eat the marker token and make the guards look like if '' in content - review the raw file.
• Why the no-progress detector skips mutating tools: identical "success" strings from different writes are productive work; identical results from read/query tools are the genuine stuck signal. Empty/unknown tool names count conservatively.
• Why the ping-pong detector does not skip mutating tools: the same write with the same arguments is a loop even though its result says "success."
• Why call signatures are never truncated: two writes can differ only deep in their content; a truncated signature would false-match them. Signatures are compare-only and never rendered, so they cost zero prompt tokens.
• Why tool output is re-rendered untrimmed: leading/trailing whitespace in diffs, Makefile tabs, and exact compiler output are significant. Head+tail truncation (when max_tool_response_chars is set) keeps both ends because coding output puts the actionable summary at the end.
• Why failure detection distinguishes harness envelopes from generic markers: the harness saying invalid input for tool is a real failure for any tool; content starting with Error: is only a failure signal for tools that execute things. Tool failure is undecidable from content alone, so classification is by tool class.
• Why the vanished-tools note exists: models do not notice their own tool list shrinking; they improvise around missing capability instead of reporting it. Structural detection converts "hope the model notices" into "tell it."

Changelog

Latest first; full detail per version lives in archive/.

• v20.1 - Cross-harness skill loading and long-run delivery. skill_loader_tools (default skill skill_view) makes every skill checkpoint, loaded-skill tracking, and completed-batch accounting work against whichever loader a harness exposes; OpenCode <skill_content> and Hermes JSON skill results are both recognised, while discovery and mutation tools are excluded. When a skill loader and a todo tool are both present, the opening checkpoint gates the first response to skill calls only, then asks for a short outcome-based todo plan before the first mutating action. long_run_nudge_after (default 48 completed tool-response batches) adds a one-window delivery reserve that reconciles acceptance requirements and directs closeout before the action budget runs out. enable_skill_routing turns the routing instructions off without disabling skill tools or todo checkpoints. Medium's verification passage now separates observations from hypotheses, requires an independent known-good control before blaming the environment, and keeps failed acceptance checks open. Six correctness fixes: history keeps answer content after the first reasoning closer; the media-marker strip covers list-shaped system content; failure detection is no longer masked by a timing suffix in the result head; auto_disable_thinking_with_tools now wins over an inline think tag; the continuation checkpoint is a third shorter; and the recommended temperature moves to 0.70. A controlled A/B established that the first-response gate does not cause post-skill-load turns to exhaust the reasoning budget.
• v20 - Action-staged skills and bounded todo maintenance. The opening checkpoint identifies the complete match set but loads only the primary skill for the immediate action and its explicit dependencies, at most three per response; remaining matches wait until their work item becomes active. Completed skill batches move directly to a concrete non-skill action unless their results reveal an explicit dependency. Successful todo updates can activate the next direct skill chain, while a windowed four-action checkpoint reconciles stale plans from evidence and stays silent when nothing changed. Completion accounting covers parallel and ID-less tool-result envelopes, error checkpoints retain priority, and compaction no longer depends on historical todo calls remaining in context. The final field run exercised three staged skill transitions and advanced todo completion from one to eight without reconciliation spam.
• v19 - Correctness release. Fixes six history-rendering defects, several shared with upstream. Reasoning is no longer rendered twice when a client sends both an explicit reasoning field and content that opens with an inline think block; message.reasoning is read as well as reasoning_content and thinking, following vLLM's rename (reasoning_content still wins when a client sends both); a single-line <think>...</think> is now extracted instead of leaking the raw tag into history; tool-call arguments that are neither a mapping nor a string are serialized rather than silently dropped; scalar parameter values render as JSON literals rather than Python reprs, where this template was the only one of the three references getting it wrong; and consecutive tool calls are separated by a single newline, matching both official Qwen 3.8 templates (this had been inherited from the v21.3 base). The separator change is justified by distribution match alone: an apparent improvement in parallel tool calling did not survive further sessions, and a leave-one-out check showed it came entirely from one anomalous session, so no behavioural gain is claimed. Adds the synthetic_tools kwarg, because a harness that invents a placeholder tool name for a schema-rejected call would otherwise trip the vanished-tools note and make the model reason about a capability loss that never happened. With thinking disabled, the tool instructions no longer tell the model to emit think blocks.
• v18 - Reasoning-effort control and bounded investigation. Makes high distinct from xhigh and gives all four tiers explicit scope: low acts one decision ahead, medium stays turn-local and bounds failed verification, high validates material alternatives without open-ended exploration, and xhigh remains comprehensive. Detailed deliverables move directly into tool calls instead of being drafted twice in reasoning. Opening delegation is decision-sized with a concrete stop condition and an empty-set option; a failure domain discovered mid-work can trigger a newly relevant skill without another user message. During the beta campaign, research web calls fell from 61 to 7 in the comparable bounded run, parent reasoning fell from 752,598 to 394,942 characters, and the deliverable-boundary test reduced reasoning on a substantial write from 33,842 to 129 characters while increasing the emitted payload. The final low-tier run had a 168-character median across 77 parent build responses with no truncations, while its opening outliers motivated low's final immediate-decision rule.
• v17 - Qwen 3.8 support. Adds reasoning_effort steering (xhigh / high alias / medium / low), defaulting to medium so no steering text is injected: in a controlled A/B on Qwen3.8-27B, Qwen's own xhigh default averaged 30,102 characters of reasoning below 40k context and truncated two outputs, one of them the harness's own conversation summary. Unknown values degrade to medium instead of raising, and steering is suppressed when thinking is off. Adds a preserve_reasoning alias so llama.cpp's --reasoning-preserve reaches the template, and merges consecutive leading system/developer messages instead of reading only messages[0]. Across two field sessions (252 requests, peak 183k context) there were no truncations, stalls, or loop warnings; conversation compaction completed cleanly where the stock template had truncated it; and mean reasoning per response fell 62%. Every detector was additionally verified against the live llama.cpp renderer. Verified as already present from earlier releases and therefore not re-added from upstream v22: the in-content thinking parser, string tool-argument handling, and the absence of the enable_thinking=false lockdown.
• v16 - Cumulative post-v15 release. Adds browser/MCP/non-2xx failure recognition, ungates error-shaped evidence from the old short-output classifier, recognizes PowerShell null-expression partial-write risk, and extends the opening skill pass with a conditional decomposition/delegation checkpoint when an exact task or agent tool exists. Delegation is explicitly sequenced after all skill results, and a second-stage loop warning escalates when the first strategy-change checkpoint is ignored. Non-research delegation remained flat; visible planning increased, while artifact quality remains under paired validation.
• v15 - Mid-context skill rechecks. Every later genuine user request receives a compact continuation checkpoint naming successfully loaded skills. The checkpoint keeps existing guidance active and directs newly direct matches to load together without reloading earlier skills. Exact harness-owned tool-response and screenshot envelopes are excluded. Field tests passed ordinary continuation, new-domain activation, batched loading, duplicate avoidance, and synthetic-envelope controls. Complete multi-domain coverage remained inconsistent, and the later-turn no-match control was not scored before release.
• v14 - Skill triggering. On harnesses that expose a skill tool (OpenCode and similar), local Qwen models see the skill catalog but rarely load skills on their own - frontier models load them routinely. v14 adds a conditional skill gate so skills trigger the way they do with frontier models: a one-time checkpoint on the first user message (compare the task against the skill descriptions across every facet of the work, then load all direct matches together in one parallel skill-only response before any other tool call), plus a point-of-need re-check clause inside the loop-detector warnings (when work is stuck, consider a skill whose description directly matches the problem). Everything renders only when a tool named exactly skill exists - every other harness gets byte-identical v13 behavior. Developed over 16 instrumented experiment rounds; final round scored 4/4 skill-only first phases with zero clear irrelevant loads and clean no-match controls. If you prefer skills not to trigger this way, use archive/chat_template-v13.jinja - it is exactly this template without the skill-trigger changes. Feedback on skill triggering (over-loading, missed matches, harnesses where the gate misbehaves) is actively wanted - please open a discussion.
• v13 - Tool-call ID result attribution. Tool results are matched to their originating call by tool_call_id, so parallel batches classify each result against the correct tool class (a failed write next to a successful read no longer cross-contaminates warnings). Fixes false failure warnings on parallel reads and lost mutating exemptions on parallel writes.
• v12 - Two-sided turn management. Added a continuation rule (keep working until complete; verify then move on; prefer small concrete actions over long deliberation) and replaced the every-turn stop-invitation with bounded exit conditions. Tuned across field sessions to avoid both premature stops and reasoning spirals - the single highest-gain wording lever in the template.
• v11 - Language release. Research-driven rewording of every model-visible phrase (negation backfires - "pink elephant" effect; neutral tone; absolutes only where literally true): warnings now let the model self-classify (expected failure mid-fix vs. real failure), consecutive-error warnings split on evidence (identical repeated error vs. changing errors/burn-down), the no-progress warning gained a polling escape hatch, and all instructions are positively framed. A permanent negation-audit test keeps prohibitive phrasings from returning.
• v10 - Truthfulness release. Read-tool exemption for content-based failure markers (read_tools kwarg); <arguments_json> compatibility wrapper for renderers without C++ argument normalization, with max_tool_arg_chars truncation; argument-grounding rules in the tool instructions; vanished-tools note reworded to allow safe continuation with remaining tools; ping-pong warning reworded to remove a false absolute and add a polling escape hatch; template comments removed (design rationale moved to this README).
• v9 - Two new structural detectors, both field-validated: ping-pong/action-loop (call-signature repeats catch loops whose results differ every turn) and vanished-tools (a tool called in history but missing from tools[] gets a system note naming it). New kwargs pingpong_nudge_after, warn_vanished_tools.
• v8 - Media-marker correctness: strip only from system/assistant/reasoning; preserve user/tool markers (a v3/v7 unconditional strip silently broke real vision and tool screenshots).
• v7 - Runtime hardening: repeat_nudge_after=0 fix, tool-output fidelity, long-error detection, head+tail truncation. Added the test cases and repro script.
• v6 - Cross-harness mutating_tools union (16+ harnesses), case-insensitive. Zero prompt-token cost (never rendered).
• v5 - Loop-detector rework on 381 real sessions: removed the noisy same-tool-streak detector; no-progress detector now skips mutating tools. ~90% fewer nudges, every real catch kept.
• v4 - Tool-envelope unwrapping (~6.5k tokens saved at 163 tools); self-healing spilled tool calls; think_on_tool_failure kwarg; emoji-free warnings.
• v3 - Media-marker injection guard (fixes ik_llama tokenize 400 from hallucinated/fetched markers).
• v2 - Date awareness via strftime_now (guarded; no-op on runtimes without it).
• v1 - Initial release: froggeric v21.3 base + loop detectors + error markers + kwargs.

Credits and license

• Base template: froggeric/Qwen-Fixed-Chat-Templates v21.3. All upstream fixes are inherited; please report template-core issues upstream.
• Agentic layer: Moore2877, developed with AI assistance. Report agentic-layer issues in this repository.
• Thanks: to froggeric and Chromix for the upstream work this release builds on.
• License: Apache-2.0.