JC1DA/Qwen3.8-27B-heretic-ara-W4A16

4-bit weight quantized version of trohrbaugh/Qwen3.8-27B-heretic-ara using AutoRound.

Quantization Details

• Scheme: W4A16 (4-bit weights, 16-bit activations)
• Format: auto_round
• Library: AutoRound
• Iterations: 1000
• Torch compile: enabled
• Group size: 128
• Ignored layers: embed_tokens, visual layers, mtp layers, input_layernorm, post_attention_layernorm, q_norm, k_norm, conv1d, linear_attn.norm

Original quantization command

auto-round --model trohrbaugh/Qwen3.8-27B-heretic-ara --scheme "W4A16" --format "auto_round" --output_dir ./Qwen_Qwen3.8-27B-INT4-W4A16-Autoround --iters 1000 --enable_torch_compile --ignore_layers "model.language_model.embed_tokens,model.visual.*,mtp.*,input_layernorm,post_attention_layernorm,q_norm,k_norm,conv1d,linear_attn.norm"

Usage

Load with AutoRound's loader:

from autoround.autoround_utils import load_model
model = load_model(model_path="JC1DA/Qwen3.8-27B-heretic-ara-W4A16")

Or with standard transformers (if compatible with your setup):

from transformers import AutoModelForCausalLM, AutoProcessor
model = AutoModelForCausalLM.from_pretrained("JC1DA/Qwen3.8-27B-heretic-ara-W4A16", trust_remote_code=True)
processor = AutoProcessor.from_pretrained("JC1DA/Qwen3.8-27B-heretic-ara-W4A16")

Model Info

• Parameters: ~27B
• Sharded: Yes (10 shards)
• Files: model-00001-of-00010.safetensors through model-00010-of-00010.safetensors