⚡ Huihui-Qwopus3.5-27B-v3-abliterated — HLWQ CT INT4

CompressedTensors INT4 of Jackrong/Qwopus3.5-27B-v3 (abliterated) via HLWQ (Hadamard-Lloyd Weight Quantization)

Native vLLM. Marlin kernel. Zero plugin. 168 tok/s on A100.

📊 Compression

🏎️ Quick Start — One Command

pip install vllm
vllm serve caiovicentino1/Huihui-Qwopus3.5-27B-v3-abliterated-HLWQ-Q5 \
  --language-model-only --enforce-eager

No plugin. No pip install polarquant. No custom code.

📊 Speed

🎯 Quality

HLWQ produces better INT4 weights than direct quantization — 0.12 PPL improvement from Hadamard rotation + Lloyd-Max preprocessing.

💻 GPU Compatibility

🧬 Architecture

🔬 How HLWQ Works

Standard INT4 quantizes weights directly — outliers cause high error. HLWQ adds a preprocessing step before INT4:

BF16 weights
    │
    ▼
[1] Hadamard rotation → distributes energy uniformly
    │  (eliminates outliers, weights become Gaussian)
    │
    ▼
[2] Lloyd-Max Q5 → MSE-optimal 5-bit quantization
    │  (best possible codebook for Gaussian distribution)
    │
    ▼
[3] Dequant → BF16 → INT4 symmetric (gs=128)
    │  (cleaner weights = better INT4)
    │
    ▼
CompressedTensors (Marlin kernel) → vLLM serve

Same speed as GPTQ/AWQ, better quality.

🔧 Usage with Transformers

pip install polarquant

import polarengine_vllm  # auto-registers with transformers
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "caiovicentino1/Huihui-Qwopus3.5-27B-v3-abliterated-HLWQ-Q5",
    device_map="auto", trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(
    "caiovicentino1/Huihui-Qwopus3.5-27B-v3-abliterated-HLWQ-Q5",
    trust_remote_code=True
)

inputs = tokenizer("Hello!", return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(out[0], skip_special_tokens=True))

⚠️ Important Flags

📖 Citation

@misc{hlwq2026,
  title={HLWQ: Hadamard-Lloyd Weight Quantization for Large Language Models},
  author={Caio Vicentino},
  year={2026},
  url={https://arxiv.org/abs/2603.29078}
}

🔗 Links