Qwen3.6-27B-int4-AutoRound

This is an Int4 AutoRound quantization of Qwen/Qwen3.6-27B, produced using spark-auto-round.

Quantization Details

Layers Kept in FP16

The linear_attn.in_proj_a and linear_attn.in_proj_b projections across all DeltaNet layers, as well as mtp.fc, are kept at FP16 precision for quality preservation.

Quantization Report

All 64 transformer blocks passed sensitivity analysis (63 PASS, 1 WARN at layer 58).

Full per-layer reports are available in the repository: quantization-report.txt and quantization-report.csv.

How to Use

With vLLM

vllm serve coder3101/Qwen3.6-27B-int4-AutoRound

With Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "coder3101/Qwen3.6-27B-int4-AutoRound"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")

prompt = "Explain the theory of relativity in simple terms."
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Acknowledgments

• Quantization performed using spark-auto-round by @whpthomas
• Based on AutoRound by Intel

---

Original Model Card -- Qwen3.6-27B

Below is the original model card from Qwen/Qwen3.6-27B.

Qwen3.6-27B

Highlights

Qwen3.6-27B follows the Qwen3.5 series with key upgrades:

• Agentic Coding: Handles frontend workflows and repo-level reasoning with greater fluency.
• Thinking Preservation: New option to retain reasoning context from historical messages, reducing overhead in iterative development.

Model Architecture

Gated DeltaNet: 48 linear attention heads for V, 16 for QK (head dim: 128) Gated Attention: 24 heads for Q, 4 for KV (head dim: 256, RoPE dim: 64)

Benchmark Results -- Language

Benchmark Results -- Vision Language

Serving Frameworks

• SGLang (>=0.5.10)
• vLLM (>=0.19.0)
• KTransformers
• HuggingFace Transformers

Sampling Parameters

Key Features

• Thinking mode is on by default; can be disabled via enable_thinking: False.
• Does not support soft switch (/think and /nothink from Qwen3).
• Preserve Thinking: preserve_thinking: True retains reasoning traces from history.
• Supports text, image, and video inputs.
• Multi-Token Prediction (MTP) supported.
• Native context length: 262,144 tokens; extensible to 1,010,000 tokens with YaRN RoPE scaling.

Citation

@misc{qwen3.6-27b,
    title  = {{Qwen3.6-27B}: Flagship-Level Coding in a {27B} Dense Model},
    author = {{Qwen Team}},
    month  = {April},
    year   = {2026},
    url    = {https://qwen.ai/blog?id=qwen3.6-27b}
}