Qwen3.6-27B PrismaQuant Heretic 5.25bit vLLM

This is a PrismaQuant mixed-precision compressed-tensors export of llmfan46/Qwen3.6-27B-uncensored-heretic-v2 for vLLM serving.

Quantization

• Target: allocator kneedle point at 5.25 bpp
• Achieved allocator bpp: 5.249 over quantizable body Linears
• Formats: NVFP4, FP8_E4M3, BF16 fallback
• Body assignment counts: 203 NVFP4, 84 FP8_E4M3, 17 BF16
• Export histogram: 285 NVFP4 production-cache Linears, 178 FP8_E4M3 production-cache Linears, 33 BF16 Linears
• Visual components: BF16 passthrough
• Source dtype: BF16

Validation

Validated locally with vLLM load and greedy generation:

• Eager mode: passed
• Graph/compiled CUDA capture mode: passed
• Prompt: The capital of France is
• Output prefix: Paris.

The serving run selected:

• CutlassFP8ScaledMMLinearKernel for compressed-tensors FP8
• FlashInferCutlassNvFp4LinearKernel for NVFP4 GEMM

No KL quality claim is made in this model card.

Serving

vllm serve rdtand/Qwen3.6-27B-PrismaQuant-Heretic-5.25bit-vllm \
  --quantization compressed-tensors \
  --trust-remote-code

For a low-context smoke:

python -m vllm.entrypoints.openai.api_server \
  --model rdtand/Qwen3.6-27B-PrismaQuant-Heretic-5.25bit-vllm \
  --quantization compressed-tensors \
  --trust-remote-code \
  --max-model-len 528

Provenance

• Source repo: llmfan46/Qwen3.6-27B-uncensored-heretic-v2
• Source revision: bf309da6e5744ac57240ae21d31eaf107734c585
• Quantizer: PrismaQuant
• Local export path: /home/rob/dq-runs/qwen36-27b-heretic-v2-kneedle-5p25-materialize-20260515T220544Z/exported