中文Sign up

rdtand/Qwen3.6-27B-PrismaQuant-Heretic-5.25bit-vllm

🤗 Hugging Face sourceimage-text-to-textapache-2.018.9B params22 GBsafetensors✓ 6 checksumsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo rdtand/Qwen3.6-27B-PrismaQuant-Heretic-5.25bit-vllm ./model-folder
Needs a seeder →

Qwen3.6-27B PrismaQuant Heretic 5.25bit vLLM

This is a PrismaQuant mixed-precision compressed-tensors export of llmfan46/Qwen3.6-27B-uncensored-heretic-v2 for vLLM serving.

Quantization

  • Target: allocator kneedle point at 5.25 bpp
  • Achieved allocator bpp: 5.249 over quantizable body Linears
  • Formats: NVFP4, FP8_E4M3, BF16 fallback
  • Body assignment counts: 203 NVFP4, 84 FP8_E4M3, 17 BF16
  • Export histogram: 285 NVFP4 production-cache Linears, 178 FP8_E4M3 production-cache Linears, 33 BF16 Linears
  • Visual components: BF16 passthrough
  • Source dtype: BF16

Validation

Validated locally with vLLM load and greedy generation:

  • Eager mode: passed
  • Graph/compiled CUDA capture mode: passed
  • Prompt: The capital of France is
  • Output prefix: Paris.

The serving run selected:

  • CutlassFP8ScaledMMLinearKernel for compressed-tensors FP8
  • FlashInferCutlassNvFp4LinearKernel for NVFP4 GEMM

No KL quality claim is made in this model card.

Serving

vllm serve rdtand/Qwen3.6-27B-PrismaQuant-Heretic-5.25bit-vllm \
  --quantization compressed-tensors \
  --trust-remote-code

For a low-context smoke:

python -m vllm.entrypoints.openai.api_server \
  --model rdtand/Qwen3.6-27B-PrismaQuant-Heretic-5.25bit-vllm \
  --quantization compressed-tensors \
  --trust-remote-code \
  --max-model-len 528

Provenance

  • Source repo: llmfan46/Qwen3.6-27B-uncensored-heretic-v2
  • Source revision: bf309da6e5744ac57240ae21d31eaf107734c585
  • Quantizer: PrismaQuant
  • Local export path: /home/rob/dq-runs/qwen36-27b-heretic-v2-kneedle-5p25-materialize-20260515T220544Z/exported