Sign up

sakamakismile/Huihui-Nex-N2-mini-abliterated-text-MTP-NVFP4

🤗 On Hugging Facetext-generationapache-2.020B params21 GBsafetensors✓ Checksum-verifiedupdated 0d ago
Magnet

Huihui-Nex-N2-mini-abliterated-text-MTP-NVFP4

Text-only NVFP4 (W4A4) quantization of huihui-ai/Huihui-Nex-N2-mini-abliterated — the abliterated Nex-N2-mini (35B-A3B qwen3_5_moe hybrid-linear MoE). This is the language model only (Qwen3_5MoeForCausalLM, vision tower dropped) at ~22.7 GB, with the native MTP draft in MTP/. For the full vision-language version, see Huihui-Nex-N2-mini-abliterated-MTP-NVFP4.

Made by quantizing the huihui-ai abliterated release with llm-compressor + compressed-tensors. All credit for the model itself goes to huihui-ai (abliteration) and nex-agi (the original Nex-N2-mini); this repo only adds the FP4 quantization.

Lineage: nex-agi/Nex-N2-mini → huihui-ai abliteration → Lna-Lab NVFP4 W4A4, text-only (this repo).

What it is

| | |

|---|---|

| Architecture | Qwen3_5MoeForCausalLM (qwen3_5_moe_text) — language model only, no vision tower |

| Total / active | ~34B total · ~3B active (256 experts, top-8, + shared expert) |

| Attention | hybrid: linear-attention (GatedDeltaNet-style SSM) ×3 → full-attention every 4th layer (40 layers) |

| MTP | native draft shipped in MTP/ (BF16, ~1.69 GB) |

| Quantization | NVFP4 nvfp4-pack-quantized, W4A4, group size 16 — 30,720 expert projections + attention/linear-attn projections packed; router/norms/conv/lm_head BF16 |

| Size | ~22.7 GB |

| Nature | abliterated / uncensored · reasoning model (…) |

⚠️ Serving note (read this)

As of vLLM 0.22.0, the standalone text config (model_type: qwen3_5_moe_text / Qwen3_5MoeForCausalLM) is not yet served directly — vLLM routes this architecture through its Qwen3-VL path and expects a multimodal config with vision_config, so it stops with a config-type mismatch. Until vLLM adds a standalone text path for qwen3_5_moe, run the vision-language sibling with --limit-mm-per-prompt '{"image":0,"video":0}' for identical text-only behavior (the language weights are byte-for-byte the same). This text-only repo is published for transformers use, future/other backends, and as the smaller checkpoint.

llm-compressor recipe: QuantizationModifier(targets="Linear", scheme="NVFP4", ignore=["lm_head","re:.visual.","re:.mlp.gate$","re:.mlp.shared_expert_gate$"]), 32 calibration samples (neuralmagic/calibration).

License

Inherits apache-2.0 from the upstream models. Abliterated/uncensored: you are responsible for how you use it.

Credits

Support the Base Model Author (huihui-ai)

If you find the abliterated base useful, please support huihui-ai — this repo only adds the FP4 quantization; the abliteration work is theirs: