中文Sign up

rdtand/Hy3-295B-A21B-PrismaQuant-5.3bit-2xSpark-vllm

🤗 Hugging Face sourceapache-2.0194B params21B activated204 GBsafetensors✓ 39 checksumsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo rdtand/Hy3-295B-A21B-PrismaQuant-5.3bit-2xSpark-vllm ./model-folder
Needs a seeder →

Hy3 295B-A21B — PrismaQuant 5.3-bit mixed NVFP4/FP8/BF16 for 2× DGX Spark

Tencent Hy3 (July 2026 production release, Apache 2.0) — 295B total / 21B active MoE, 80 layers, 192 routed experts top-8, including the MTP draft layer for speculative decoding — quantized by PrismaQuant from the BF16 source to a ~190 GB compressed-tensors checkpoint (5.28 bits/param over quantizable weights) sized to serve on two NVIDIA DGX Sparks under vLLM pipeline parallelism with no custom kernels and no forked runtime.

Sibling artifact: the single-Spark 2.8-bit GGUF is at rdtand/Hy3-295B-A21B-PrismaQuant-2.8bit-gguf-vllm.

What's inside

  • Measured mixed-precision allocation (per-tensor formats from a knapsack solve over measured per-(tensor, format) error weighted by an empirical Fisher probe of the BF16 model): NVFP4 on the routed-expert bulk, FP8 (E4M3, dynamic activations) on most attention/shared tensors, BF16 on the small sensitive remainder. Full map in allocation/layer_config.json.
  • Byte-budget selection: sized so weights fit 2×128 GB with KV headroom (~99 GB/node under pp2), not tuned to a curve heuristic.
  • Deliberate rendering throughout: GPTQ + static activation ordering + joint scale optimization on dense Linears; batched GPTQ on every one of the 15,360 routed experts (rendered inline during the streaming export — the model never fits in memory at any point in the pipeline).
  • MTP included: model.layers.80 ships verbatim BF16 and loads into vLLM's HYV3 MTP path (--speculative-config '{"method":"mtp",...}').
  • Embeddings/head BF16; norms/router FP32.

Validation — read this

No quality claims are made for this artifact, and the author has one Spark: the full 2-node topology has not been run. What was validated:

  • A 6-layer truncated twin of this exact artifact (same quantization metadata, real tensors, MTP remapped) passes vLLM load + generate and MTP speculative-decode smokes on a single Spark. pp2 loads each rank through the same unsharded per-layer path the twin exercises.
  • A static tp2 shard audit (shipped: serving/tp2_shard_audit.py) verifies every quantized tensor's packed dims, NVFP4 group-16 boundaries, and scale shapes divide cleanly at world size 2.
  • Per-tensor packing is bit-exact against the pipeline's own emulation.

Community measurements (2×Spark throughput, KL/perplexity vs BF16, task suites) are welcome and will be linked here.

Serving

See serving/README.md for the two-node pp2 launch (recommended) and the tp2 alternative.

Provenance

Quantized with PrismaQuant. Source: tencent/Hy3 (BF16). Allocation and Pareto data in allocation/. Contact: robert.tand@icloud.com