中文Sign up

sahilchachra/Qwen3.6-35B-A3B-uncensored-heretic-AWQ

🤗 Hugging Face sourcetext-generationapache-2.035.4B params3B activated136 GBsafetensors✓ 6 checksumsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo sahilchachra/Qwen3.6-35B-A3B-uncensored-heretic-AWQ ./model-folder
Needs a seeder →

Qwen3.6-35B-A3B-uncensored-heretic-AWQ (W4A16)

AWQ 4-bit (W4A16) quantization of llmfan46/Qwen3.6-35B-A3B-uncensored-heretic — an uncensored ("heretic"/abliterated) variant of Qwen3.6-35B-A3B (qwen3_5_moe: a hybrid GatedDeltaNet (linear-attention) + full-attention Mixture-of-Experts with 256 routed experts (8 active) + a shared expert, plus a vision tower and an MTP head).

Variant: AWQ W4A16 — 4-bit symmetric integer weights, group size 128, with activation-aware scaling. Activations stay BF16. Disk size: ~21.8 GB (vs ~70 GB BF16, ~3.2×) Quantized by: sahilchachra Tooling: llm-compressor 0.12 (AWQModifier + QuantizationModifier) → compressed-tensors pack-quantized

Why this source: the popular HauhauCS/Qwen3.6-35B-A3B-Uncensored-* repos ship only GGUF (no full-precision weights), which cannot be honestly AWQ-quantized. This checkpoint quantizes a genuine BF16 uncensored Qwen3.6-35B-A3B instead.

What is quantized

Quantized to int4 (30,880 modules):

  • routed experts mlp.experts.{0..255}.{gate,up,down}_proj on all 40 layers — 30,720
  • shared expert mlp.shared_expert.{gate,up,down}_proj on all 40 layers — 120
  • full-attention self_attn.{q,k,v,o}_proj on the 10 full-attention layers — 40

Kept in BF16 (not quantized): the GatedDeltaNet linear_attn (mamba) layers, the MoE router (mlp.gate) and shared_expert_gate, the vision tower, the MTP head, token embeddings, lm_head, and all norms.

Calibration

AWQ is calibration-based. Calibrated on 32 sequences × 512 tokens of general instruction-following chat from HuggingFaceH4/ultrachat_200k, rendered through the model's chat template. Because this is the base instruct model with refusals abliterated (not a task-specific fine-tune), general chat best matches its inference distribution. During calibration every one of the 256 experts sees all calibration tokens (moe_calibration_context), so per-expert AWQ scales are well-estimated despite sparse 8/256 routing.

Loading / serving

This checkpoint stores experts in the per-expert compressed-tensors (pack-quantized) layout that vLLM consumes for qwen3_5_moe. Serve with vLLM:

from vllm import LLM, SamplingParams
llm = LLM(model="sahilchachra/Qwen3.6-35B-A3B-uncensored-heretic-AWQ",
          trust_remote_code=True, dtype="bfloat16")
print(llm.generate(["Write a quicksort in Python."], SamplingParams(max_tokens=256))[0].outputs[0].text)

Notes

  • Text-only: the source repo nests its vision-tower weights under a non-standard key prefix (model.language_model.visual.* instead of model.visual.*), so the vision tower does not load functionally. This is a quantization of the language model; treat it as a text-generation model.
  • GatedDeltaNet kernels (causal-conv1d / flash-linear-attention) fall back to a pure-PyTorch path where unavailable (e.g. ARM/Jetson) — correct, just slower.
  • compressed-tensors == 0.17.1's eager decompress_model mis-reads group_size for pack-quantized checkpoints; load via vLLM (its own loader) or a fixed compressed-tensors. The weights/scales on disk are standard and correct.

Uncensored

This is an abliterated / uncensored model (refusal directions removed). It will follow instructions it would otherwise decline. Use responsibly and in compliance with applicable law and the Apache-2.0 / Qwen terms.

Original model

See llmfan46/Qwen3.6-35B-A3B-uncensored-heretic and base Qwen/Qwen3.6-35B-A3B for architecture, intended use, and license.