Sign up

peculiar-ragdoll/Nail-Qwen3.6-35B-A3B-GGUF

🤗 On Hugging Faceimage-text-to-textapache-2.0116 GBGGUF✓ HF checksums availableupdated today
Magnet

Straight to the point

Nail dominates Qwen3.6-27b and its ThinkingCap fine-tune on time-to-answer while matching their accuracy on reasoning and agentic software engineering ability, and wins on multi-turn conversation quality even when the 27b models use a higher quantization.

Say goodbye to overthinking, tool call failures, amnesic loops, and fluffy outputs.

MTP version here has a larger selection of quants.

This is Unsloth's UD-Q4_K_XL quant of Qwen3.6-35B-A3B with an improved chat template also containing a force-appended system prompt that sharpens behaviour. llama.cpp applies it

automatically, nothing needed from you.

If you have a Mac, use the MLX build instead.

This exists for everyone who doesn't.

The numbers

All benchmarks on Mac Studio M2 Ultra 64 GB, oMLX, 8bit KV cache, MLX quants.

Reasoning and knowledge

Multi turn conversations, capped at 6 turns

Autonomous software engineering in Pi coding agent

Best conversation score, fastest to a working fix, fastest to a correct answer — against dense models that take 2-5x times as long, at accuracy parity with all of them. Four benchmarks, n=3 seeds, one Mac Studio M2 Ultra. Nail beats Opus4.8 (medium) 3-0 on first-attempt solves in a real SWE Live repo, no fluffing about. We used Pi for local models, Claude Code for Opus.

Which one. Many independent tasks, or you'd rather solve 3 problems in the time Qwen3.6-27b solves one → Nail. One long agentic session that has to stay coherent inside a single context → Dagger-27b, its dense sibling at the same ~21 GB RAM footprint.

Deployment math. The Nail GGUF and about 180k tokens of context in 8-bit KV should fit comfortably on a 24GB RAM card, which means it probably runs 120 tok/s on a single RTX4090 or RX7900 without partial offloading.

Memory footprint in multi-turn conversations to the context ceiling

Nail with full context and unquantized KV cache fits and runs on a 32GB unified RAM Mac, because it uses less RAM per token in context even at full 16bit KV precision.

While Nail runs all the way to 92 turns and 262k tokens context on that machine, Dagger taps out at 42 turns and 73k tokens.

Constrained hardware makes Nail the marathon winner AND the sprint winner, without sacrificing quality over Dagger or ThinkingCap.

Dagger wins for users with a lot of RAM, and a very long task, who are not in a hurry.

The other axis: context

Long form stamina is context ceiling ÷ tokens-per-question:

Seconds-per-correct is a sprint metric. In a long session that keeps every turn's thinking in context, the binding resource isn't time — it's context space, and your ceiling is context ÷ tokens per question.

Nail declares the same 262,144-token native context as Dagger, so the ceiling is not where it loses — the fill rate is. On GPQA-Diamond Nail spends 5,777 tokens per question against Dagger's 2,380:

  • Hard questions chained inside the native context: Dagger 110 · ThinkingCap 63 · Nail 45 · stock Qwen 24.
  • Conversation turns inside 100k tokens, on a harness that keeps each turn's thinking (Claw-Eval per-turn think+answer): Dagger 57 · ThinkingCap 44 · Nail 35.

Nobody sat through a 110-step session — it matters for autonomous thinking, agentic work, and conversations you keep coming back to.

Nail's verbose thinking is free in seconds and expensive in context. It still beats stock Qwen on both, but here Dagger wins by 2.4×. Nail for throughput, Dagger for endurance — the two axes have opposite winners, and neither model answers both.

The secret sauce

The system prompt ships inside the chat template and applies on every call. If you send your own system prompt, yours is emitted first and Nail's is appended after it:

You are Nail-35b-a3b, a variant of Qwen3.6-35B-A3B. Answer directly, after thinking. Lead with the answer, then only what it needs to be correct and usable.
Never: open with preamble or pleasantries; restate the question; add filler transitions; hedge with niceties; or repeat a point you've already made.
Always: keep essential steps, caveats, uncertainties, and specifics — never drop correctness or a needed warning for brevity. Keep the final answer lean. Use the least structure that conveys it (plain prose when short; lists or code only when they earn their place). If genuinely uncertain, say so and explain why — never omit uncertainty for the sake of brevity.
If a user request is genuinely ambiguous, ask a sharp question, don't guess.

Byte-identical to Dagger's, apart from the name.

The prompt is always on and lives in the template, not the API — disabling it means editing chat_template.jinja.

Froggeric's chat template that we used as a base implements many tricks that drastically improves multi-turn agentic workflows with tool calling.

The composition of our battle tested prompt with the improved template is what makes Nail so effective.

What the numbers are, and aren't

Every measurement we publish was taken on the MLX build, not on this file. They are not the same

artifact: MLX Unsloth-Dynamic-4bit and GGUF UD-Q4_K_XL are different quantization schemes at a

similar size, and llama.cpp and oMLX are different runtimes. On the one head-to-head we did run —

the UD-Q4_K_XL file, llama-bench on an M2 Ultra, 2k prompt — llama.cpp was **1.6× faster on

prefill and level-to-slightly-slower on decode** than oMLX on the MLX build. The Q5 and Q6

files are unmeasured; expect slower decode roughly in proportion to their size.

So: treat the plate as evidence about the recipe — a 3.4B-active MoE with this template and this

prompt — not as a measurement of this file. If you want numbers for this exact GGUF, we haven't

taken them.

Use

Let llama.cpp fetch it — pass a :quant tag (:Q4_K_S, :Q4_K_XL, :Q5_K_XL, :Q6_K_XL, :IQ3_S). The tag

is required: this repo has no Q4_K_M, so a bare -hf with no tag falls back to the wrong file. The

mmproj rides along in the manifest, so vision works from the same tag — no second download.

# text — auto-downloads to llama.cpp's own cache
llama-server   -hf peculiar-ragdoll/Nail-Qwen3.6-35B-A3B-GGUF:Q4_K_XL -ngl 99   # or llama-cli
# vision — same tag; the mmproj is pulled automatically
llama-mtmd-cli -hf peculiar-ragdoll/Nail-Qwen3.6-35B-A3B-GGUF:Q4_K_XL -ngl 99 --image photo.jpg

Prefer to keep the files yourself? Download explicitly, then point -m at the local path:

hf download peculiar-ragdoll/Nail-Qwen3.6-35B-A3B-GGUF \
  --include "*UD-Q4_K_XL.gguf" "mmproj-F16.gguf" --local-dir Nail-GGUF   # a bare download pulls ~116 GB
llama-cli      -m Nail-GGUF/Nail-Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf -ngl 99                          # text
llama-mtmd-cli -m Nail-GGUF/Nail-Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf \
               --mmproj Nail-GGUF/mmproj-F16.gguf -ngl 99 --image photo.jpg                        # vision

Which file

| file | size | notes |

|---|--:|---|

| Nail-Qwen3.6-35B-A3B-UD-IQ3_S.gguf | 13.7 GB | smallest — fits a 16 GB VRAM card (the MoE's cheap KV leaves room for context at this weight size). Lowest quality of the tiers; use a larger quant if it fits |

| Nail-Qwen3.6-35B-A3B-UD-Q4_K_S.gguf | 20.9 GB | the roomy 24 GB pick — a hair smaller than Q4_K_XL, trading a sliver of quality for noticeably more context headroom on a 24 GB card. Start here if you want the most room to breathe at 4-bit |

| Nail-Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf | 22.4 GB | the measured 4-bit — the file the head-to-head below was run on, and the size-match to a 27B at 6-bit. Reach for this over Q4_K_S when you have the room and want the exact file the numbers came from |

| Nail-Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf | 26.6 GB | more bits if you have the headroom |

| Nail-Qwen3.6-35B-A3B-UD-Q6_K_XL.gguf | 31.8 GB | most bits. On a 64 GB machine the MoE's cheap context makes this affordable where a dense 27B at the same quality would not be |

All five are Unsloth Dynamic quants, all carry the identical embedded template, and all use the

same mmproj-F16.gguf for vision — you only need one copy of it. All five are -hf tag

addressable, e.g. llama-cli -hf peculiar-ragdoll/Nail-Qwen3.6-35B-A3B-GGUF:UD-Q6_K_XL.

Your own system prompt is preserved — the terseness directives are appended after it, so a harness

with a large prompt of its own still gets its instructions honored. Override the concision

directives explicitly and you have plain Qwen3.6-35B-A3B with a fixed template.

Recommended sampling

temperature 1.0 · top_p 0.95 · top_k 20 · min_p 0

No repetition or presence penalties. Concision comes from the prompt; penalizing tokens distorts

thinking in ways we haven't tested.

Limitations

  • The numbers above are not from this file, but from MLX. See above. This is the honest caveat.
  • The prompt is not removable through the API. It lives in the GGUF's embedded chat template.

Use Unsloth's original if you need

unmodified behavior.

  • Vision needs mmproj-F16.gguf and llama-mtmd-cli. Plain llama-cli is text-only.
  • We did not quantize this. Quality is entirely Unsloth's UD recipe; we changed one metadata

string. Any quantization loss is theirs to characterize, and they document it better than we could.

Credits

  • Qwen at Alibaba — the Qwen3.6-35B-A3B base model.
  • Unsloth — the UD-Q4_K_XL quantization this repo redistributes.
  • froggeric — the fixed Qwen chat template.
  • llama.cpp — the runtime.

Citation

@misc{Nail-35B-A3B-GGUF,
  title  = {Nail-Qwen3.6-35B-A3B-GGUF},
  author = {Saga Ishtardottir},
  year   = {2026},
  url    = {https://huggingface.co/peculiar-ragdoll/Nail-Qwen3.6-35B-A3B-GGUF},
  note   = {Unsloth's UD-Q4_K_XL quant of Qwen3.6-35B-A3B with a fixed chat template and an always-on terseness prompt}
}

License

Apache-2.0, inherited from Qwen3.6-35B-A3B.