Huihui-Qwopus3.5-27B-v3-abliterated-NVFP4

NVFP4 quantized version of huihui-ai/Huihui-Qwopus3.5-27B-v3-abliterated — an abliterated Qwen 3.5 27B distilled from Claude Opus reasoning patterns (Jackrong/Qwopus3.5-27B-v3).

~51 GB → 19.8 GB. Single NVIDIA Blackwell GPU.

What Makes This Different

Qwopus = Qwen 3.5 + Opus-style reasoning. The base model was trained to produce stable, structured reasoning even at short output lengths — where vanilla Qwen3.5 needs more tokens to reach peak quality.

| Characteristic | Vanilla Qwen3.5 | Qwopus |

|---|---|---|

| Short responses (64-256 tok) | Quality ramps up slowly | Stable from the start |

| Long responses (2048+ tok) | Slightly higher peak | Marginally lower peak |

| Reasoning style | Variable | Structured "thinking process" chains |

| Best for | Long-form analysis | Agents, tool calls, code generation |

This makes Qwopus especially suited for agentic workflows where each tool call or code snippet needs to be precise in a short response.

Key Specs

| | |

|---|---|

| Base model | huihui-ai/Huihui-Qwopus3.5-27B-v3-abliterated |

| Architecture | Qwen 3.5 Dense — 27B parameters, 64 layers |

| Quantization | NVFP4 W4A4 (weights FP4, activations FP4, scales FP8) |

| Format | compressed-tensors (native vLLM support) |

| Tool | vllm-project/llm-compressor (main) |

| Calibration | 512 samples, neuralmagic/calibration, seq_len=4096 |

| Size | 19.8 GB |

| Max context | 262,144 tokens |

| MTP | Not available (removed during fine-tuning) |

| Requires | NVIDIA Blackwell GPU (SM 120), vLLM nightly (cu130) |

Quickstart

vLLM

vllm serve Lna-Lab/Huihui-Qwopus3.5-27B-v3-abliterated-NVFP4 \
    --max-model-len 32768 \
    --reasoning-parser qwen3

With tool calling

vllm serve Lna-Lab/Huihui-Qwopus3.5-27B-v3-abliterated-NVFP4 \
    --max-model-len 32768 \
    --reasoning-parser qwen3 \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_xml

Docker

docker run --gpus '"device=0"' -p 8016:8016 \
    -v /path/to/model:/models/current:ro \
    --shm-size 16gb \
    vllm/vllm-openai:cu130-nightly \
    vllm serve /models/current --port 8016 --max-model-len 32768 \
    --reasoning-parser qwen3

Benchmark

Single NVIDIA RTX PRO 6000 Blackwell (96 GB VRAM).

| Test | Tokens | Speed | Result |

|------|--------|-------|--------|

| Short (derivative) | 64 | 57.7 tok/s | PASS — structured reasoning even at 64 tokens |

| Short (CAP theorem) | 128 | 58.5 tok/s | PASS |

| Mid (LRU cache) | 512 | 57.6 tok/s | PASS |

| Long (architecture analysis) | 1710 | 59.2 tok/s | PASS |

| Container burst | 256 | 61.1 tok/s | PASS — 3 runs stable |

Sustained: ~59-61 tok/s (single GPU).

Quantization Details

Recipe

recipe = QuantizationModifier(
    targets=["Linear"],
    ignore=["lm_head", "re:.*visual.*", "re:.*in_proj_a$", "re:.*in_proj_b$"],
    scheme="NVFP4",
)

Calibration

• Dataset: neuralmagic/calibration (LLM split)
• Samples: 512
• Max sequence length: 4096

Reproduction

from transformers import Qwen3_5ForConditionalGeneration, AutoProcessor, AutoTokenizer
from datasets import load_dataset
from llmcompressor import oneshot
from llmcompressor.modifiers.quantization import QuantizationModifier
import torch

MODEL_ID = "huihui-ai/Huihui-Qwopus3.5-27B-v3-abliterated"
OUTPUT = "Huihui-Qwopus3.5-27B-v3-abliterated-NVFP4"

model = Qwen3_5ForConditionalGeneration.from_pretrained(MODEL_ID, dtype="auto", trust_remote_code=True)
processor = AutoProcessor.from_pretrained(MODEL_ID, trust_remote_code=True)

recipe = QuantizationModifier(
    targets=["Linear"],
    ignore=["lm_head", "re:.*visual.*", "re:.*in_proj_a$", "re:.*in_proj_b$"],
    scheme="NVFP4",
)

ds = load_dataset("neuralmagic/calibration", name="LLM", split="train[:512]")

def preprocess(example):
    messages = [
        {"role": m["role"], "content": [{"type": "text", "text": m["content"]}]}
        for m in example["messages"]
    ]
    return processor.apply_chat_template(
        messages, return_tensors="pt", padding=False, truncation=True,
        max_length=4096, tokenize=True, add_special_tokens=False,
        return_dict=True, add_generation_prompt=False,
    )

ds = ds.map(preprocess, batched=False, remove_columns=ds.column_names)

def data_collator(batch):
    assert len(batch) == 1
    return {
        key: (torch.tensor(value) if key != "pixel_values"
              else torch.tensor(value, dtype=torch.bfloat16).squeeze(0))
        for key, value in batch[0].items()
    }

oneshot(
    model=model, recipe=recipe, dataset=ds,
    max_seq_length=4096, num_calibration_samples=512,
    data_collator=data_collator,
)

model.save_pretrained(OUTPUT, save_compressed=True)
processor.save_pretrained(OUTPUT)

Note: No save_mtp_tensors_to_checkpoint needed — Qwopus does not have MTP head (removed during fine-tuning).

Environment

| Package | Version |

|---------|---------|

| torch | 2.11.0+cu130 |

| transformers | 5.5.4 |

| llmcompressor | 0.1.dev (main @ 3084520) |

| compressed-tensors | 0.15.1a20260414 |

| CUDA | 13.0 |

Requirements

• GPU: NVIDIA Blackwell (SM 120)
• VRAM: ~20 GB minimum
• Software: vLLM nightly (cu130)

Notes

• Abliterated (uncensored). Use responsibly.
• Vision tower preserved in BF16.
• No MTP head — speculative decoding not available for this variant.
• NVFP4 is Blackwell-specific. Will not work on Ampere/Hopper.

Credits

• Abliteration: huihui-ai
• Opus distillation: Jackrong/Qwopus3.5-27B-v3
• Original: Qwen
• Quantization recipe: lyf/HauhauCS (proven path)
• Quantization tool: vllm-project/llm-compressor

Support the Base Model Author

• Ko-fi: https://ko-fi.com/huihuiai
• Bitcoin: bc1qqnkhuchxw0zqjh2ku3lu4hq45hc6gy84uk70ge