Qwen3.6-35B-A3B-MTP-TQ3_4S

TurboQuant TQ3_4S GGUF release for the Unsloth Qwen3.6 35B native-MTP line.

This release is named explicitly as Qwen3.6-35B-A3B-MTP-TQ3_4S and is intended for the TurboQuant runtime fork.

Files

Size Comparison

Against the standard Unsloth UD-Q3_K_M MTP control:

• Qwen3.6-35B-A3B-MTP-TQ3_4S: 13.02 GiB
• Qwen3.6-35B-A3B-UD-Q3_K_M: 15.93 GiB
• Size delta: 2.91 GiB smaller
• Relative size: about 18.3% smaller

Runtime

This GGUF needs a TurboQuant-capable runtime for native TQ3_4S and native MTP support.

• Runtime fork: turbo-tan/llama.cpp-tq3
• Recommended binary: llama-server
• Recommended frontend: LM Studio with the TurboQuant backend swap described in the repo notes.
• MTP runtime note: use -np 1 for native MTP loads; multi-sequence startup trips the current single-sequence assertion.

Recommended 35B launch contract:

./build/bin/llama-server \
  -m Qwen3.6-35B-A3B-MTP-TQ3_4S.gguf \
  --chat-template-file chat_template.jinja \
  -ngl 99 -c 4096 -np 1 \
  -ctk q4_0 -ctv tq3_0 -fa on \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  --reasoning off --reasoning-budget 0 --reasoning-format deepseek

Validation

Release validation on the 35B contract (RTX 3090 24GB, ngl 99, np 1):

Native TQ3_4S artifact

• spec-off: 74/86
• spec-off speed:
  • average: 135.51 tok/s
  • median: 136.06 tok/s
  • max: 138.21 tok/s
• spec-on: 75/86
• spec-on speed:
  • average: 177.78 tok/s
  • median: 176.88 tok/s
  • max: 194.22 tok/s

Non-TQ3_4S control

Apple-to-apple control on the same runtime and the same KV contract (ctk=q4_0, ctv=tq3_0):

• Unsloth UD-Q3_K_M spec-off: 73/86 @ 132.21 tok/s
• Unsloth UD-Q3_K_M spec-on: 74/86 @ 175.31 tok/s

This means the native TQ3_4S release stays ahead of the standard Unsloth UD-Q3_K_M control by 1 point on both spec-off and spec-on, while also remaining slightly faster.

Test Status

Current release conclusion:

• 35B is in publishable shape.
• Native TQ3_4S spec-on is the best confirmed row on the release-candidate runtime.
• 27B is still useful as supporting evidence, but not the headline publish claim.
• 9B only needs a sanity smoke check; it is not blocking this 35B release.

Base Model

• Upstream base family: Qwen/Qwen3.6-35B-A3B
• Upstream GGUF source line: unsloth/Qwen3.6-35B-A3B-MTP-GGUF

Notes

• The publish comparison is intentionally centered on the 35B line.
• The release claim is about local quality and speed under the exact contract above, not about universal leaderboard rank.
• Keep --jinja enabled for chat usage so the intended chat template stays active.
• Prefer --chat-template-file chat_template.jinja for the recommended contract so the no-empty-think behavior is explicit and reproducible.
• For LM Studio, swap in the TurboQuant backend from turbo-tan/llama.cpp-tq3 and keep the session single-sequence (-np 1 equivalent).

Credits

• MTP runtime is based on llama.cpp PR #22673.