Qwopus3.6-27B-v2-TQ3_4S

TQ3_4S is a 3.5-bit Walsh-Hadamard-transform weight format with four per-8 scales per 32-weight block.

This repository packages Jackrong/Qwopus3.6-27B-v2 as a TurboQuant TQ3_4S GGUF for local deployment.

Quantization Source

• HF source checkout:
  • Jackrong/Qwopus3.6-27B-v2
• upstream family:
  • Qwen/Qwen3.6-27B
• F16/BF16 GGUF used as the quantization source:
  • Qwopus3.6-27B-v2-f16.gguf or the converted BF16 source used by llama-quantize

Quantized with:

./build/bin/llama-quantize --pure \
  /path/to/Qwopus3.6-27B-v2-f16.gguf \
  /path/to/Qwopus3.6-27B-v2-TQ3_4S.gguf \
  TQ3_4S \
  16

Runtime Validation

Validated on a TurboQuant-capable llama.cpp build.

Example server settings:

llama-server \
  -m Qwopus3.6-27B-v2-TQ3_4S.gguf \
  --host 127.0.0.1 --port 8080 \
  -ngl 99 -c 8192 -np 1 \
  -ctk q4_0 -ctv tq3_0 -fa on \
  --jinja --no-cache-prompt --cache-ram 0

Runtime

Native TQ3_4S support requires the turbo-tan/llama.cpp-tq3 fork.

• Launch contract: -ngl 99 -c 8192 -np 1 -ctk q4_0 -ctv tq3_0 -fa on
• Keep --jinja enabled for the bundled chat template
• Use the matching mmproj.gguf for multimodal input

Files

Notes

• The source repo includes mmproj.gguf in its paired GGUF repository.
• Upload mmproj.gguf alongside this model to preserve multimodal packaging.
• Keep --jinja enabled so the bundled template is used.
• If your frontend says image input is unsupported, it is usually pointing at a server instance that was started without --mmproj.
• If native TQ3_4S support is missing, use the turbo-tan/llama.cpp-tq3 fork.

Credits

• Jackrong/Qwopus3.6-27B-v2
• Qwen/Qwen3.6-27B
• turbo-tan/llama.cpp-tq3
• llama.cpp