[!IMPORTANT] Naming notice (2026-04-10). The "HLWQ" technique used in this model is being rebranded to HLWQ (Hadamard-Lloyd Weight Quantization). The change is only the name; the algorithm and the weights in this repository are unchanged.

The rebrand resolves a name collision with an unrelated, earlier KV cache quantization method also named HLWQ (Han et al., arXiv:2502.02617, 2025). HLWQ addresses weight quantization with a deterministic Walsh-Hadamard rotation and Lloyd-Max scalar codebook; Han et al.'s HLWQ addresses KV cache quantization with a random polar rotation. The two methods are technically distinct.

Existing loaders that load this repository by ID continue to work without changes. Future model uploads will use the HLWQ name.

Reference paper for this technique: arXiv:2603.29078 (v2 in preparation; v1 still uses the old name).

🧊 Gemma-4-E4B-it — HLWQ Multi (PQ1-PQ8)

ALL quantization variants of Google's hottest model (108K downloads in 2 days).

Multimodal: text + image + audio + video | Apache 2.0 | 128K context | Edge-optimized

📊 All Variants

Quality scale:
PQ1: ██░░░░░░░░ 0.788   PQ4: █████████░ 0.991
PQ2: █████████░ 0.931   PQ5: ██████████ 0.996 ← recommended
PQ3: █████████░ 0.977   PQ8: ██████████ 0.999

✅ Generation Quality (Verified)

Real generation tests comparing HLWQ output vs the original BF16 model on 3 prompts:

Summary

Recommendation: Use PQ5 (11.7 GB) for the best quality-to-size ratio. It produces outputs identical to the original BF16 model across all tested prompts.

🚀 Quick Start

import torch
from transformers import AutoTokenizer
from safetensors.torch import load_file

# Choose your variant: PQ1, PQ2, PQ3, PQ4, PQ5, PQ6, PQ8
VARIANT = "PQ5"
REPO_ID = "caiovicentino1/Gemma-4-E4B-it-HLWQ-Multi"

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(REPO_ID, trust_remote_code=True)

# Load quantized weights
from huggingface_hub import hf_hub_download

codes_path = hf_hub_download(REPO_ID, f"{VARIANT}/codes.safetensors")
bf16_path = hf_hub_download(REPO_ID, f"{VARIANT}/bf16.safetensors")

codes = load_file(codes_path)
bf16_weights = load_file(bf16_path)

print(f"Loaded {VARIANT}: {len(codes)} quantized layers + {len(bf16_weights)} BF16 layers")

Full Inference with HLWQ

# pip install polarquant
from polarquant import HLWQModel

model = HLWQModel.from_pretrained(
    "caiovicentino1/Gemma-4-E4B-it-HLWQ-Multi",
    variant="PQ5",          # Choose: PQ1-PQ8
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

tokenizer = AutoTokenizer.from_pretrained(
    "caiovicentino1/Gemma-4-E4B-it-HLWQ-Multi",
    trust_remote_code=True,
)

messages = [{"role": "user", "content": "What is 2+2?"}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(inputs, max_new_tokens=128)
    
print(tokenizer.decode(output[0], skip_special_tokens=True))

💻 Hardware Compatibility

🏗️ Architecture

Gemma-4-E4B-it (4.5B effective, 8B with PLE embeddings)
├── 42 layers, hybrid sliding window (512) + global attention
├── Multimodal: text + image (~150M) + audio (~300M) + video
├── 128K context, 262K vocabulary
├── Per-Layer Embeddings (PLE) for edge efficiency
├── Function calling + reasoning mode
└── 386 layers quantized, rest preserved in BF16

🔬 Why HLWQ > GGUF

Lloyd-Max centroids are mathematically optimal for Gaussian-distributed weights:

PQ3 (cos_sim 0.977) > GGUF Q3_K_M (~0.95)  — same size, better quality
PQ5 (cos_sim 0.996) > GGUF Q5_K_M (~0.99)  — same size, better quality

GGUF uses uniform spacing between quantization levels — suboptimal. HLWQ places centroids where the weight density is highest (Lloyd-Max algorithm). Hadamard rotation spreads outliers before quantizing — each code carries maximum information.

📦 Files

PQ1/codes.safetensors + bf16.safetensors   (1-bit, 10.1 GB)
PQ2/codes.safetensors + bf16.safetensors   (2-bit, 10.5 GB)
PQ3/codes.safetensors + bf16.safetensors   (3-bit, 10.9 GB)
PQ4/codes.safetensors + bf16.safetensors   (4-bit, 11.3 GB)
PQ5/codes.safetensors + bf16.safetensors   (5-bit, 11.7 GB) ← recommended
PQ6/codes.safetensors + bf16.safetensors   (6-bit, 12.1 GB)
PQ8/codes.safetensors + bf16.safetensors   (8-bit, 12.9 GB)
config.json + tokenizer files

🔗 Links

• 📄 Paper — arXiv:2603.29078
• 💻 GitHub — HLWQ-Engine
• 📦 PyPI — pip install polarquant
• 🏠 Original Model

📖 Citation

@article{polarquant2026,
  title={HLWQ: Hadamard-Rotated Lloyd-Max Quantization for Large Language Models},
  author={Vicentino, Caio},
  journal={arXiv preprint arXiv:2603.29078},
  year={2026}
}

---

Quantized with HLWQ — Hadamard + Lloyd-Max optimal quantization. First multi-variant HLWQ release.

🧪 PQ2-Mixed: Smallest Functional Variant

MLP in PQ2 (4 centroids) + Attention in PQ4 (16 centroids) — smaller than PQ3 pure, better quality than PQ2 pure.

Generation Test

PQ2-Mixed is the smallest variant that produces correct factual answers. Use PQ3+ for production.