SuperHY3-abliterated-NVFP4

A fully fused NVFP4 W4A16 release of Tencent Hy3, post-trained for directness while preserving measurable capability and response integrity.





SuperHY3 combines OBLITERATUS 0.1.2 abliteration with a compact,

quality-selected SuperTune post-training update. Both stages are fused into

the checkpoint: there is no adapter to load and no runtime patch to apply.

Release Highlights

| | |

|---|---|

| Base architecture | tencent/Hy3, 295B MoE / 21B active / 192 routed experts, top-8 |

| Release format | NVFP4 weight-only W4A16, approximately 181 GB |

| Fused update | 40 attention output projections across layers 39-78 |

| Routed expert changes | 0 expert tensors modified |

| 500-prompt mean | 71.6 -> 71.8 against the original Hy3 runtime |

| Response integrity | 64/64 refusal-suite responses clean; 12/12 runtime audit cases passed |

| Artifact verification | 119/119 Hub files checksum-verified; release gate passed with 0 blockers |

What this release is designed to deliver

• Fully fused behavior: OBLITERATUS and SuperTune deltas are already inside

the weights and match the bundled chat template.

• Targeted editing: only attention output projections were changed; routed

MoE expert tensors and their NVFP4 packing remain untouched.

• Measured tradeoffs: every benchmark task is published, including the two

regressions, instead of reporting only the improved scores.

• Release evidence included: benchmark rows, refusal comparison, raw-response

audit, fusion provenance, and machine-readable release-gate results ship in

this repository.

Benchmark Snapshot

The comparison uses the same preserved 500 prompts, with 100 prompts per

task, greedy direct decoding, and the same scorer on both sides. IFEval uses the

official Google instruction checker.

| Benchmark | Original Hy3 | SuperHY3 | Delta |

|---|---:|---:|---:|

| GPQA Diamond | 46.0 | 45.0 | -1.0 |

| MMLU-Pro | 66.0 | 60.0 | -6.0 |

| IFEval strict prompt accuracy | 76.0 | 80.0 | +4.0 |

| HumanEval+ pass@1 | 82.0 | 83.0 | +1.0 |

| MBPP+ pass@1 | 88.0 | 91.0 | +3.0 |

| Five-task mean | 71.6 | 71.8 | +0.2 |

Candidate invalid-response, blank-response, and thought-leak ratios were all

0.0 across the 500 items.

The behavioral comparison used an IQ2_M Hy3 runtime with the exact same 40

fused projection deltas and release chat template. It isolates the

post-training behavior, but it is not a native NVFP4-kernel throughput or

perplexity measurement.

OBLITERATUS + SuperTune

1. Direction discovery: OBLITERATUS 0.1.2 built difference-of-means refusal

directions from 32 paired prompts.

2. Validated abliteration: the final release applies the selected

attention-output directions to layers 39-77.

3. Quality recovery: a rank-1 update, orthogonalized against the refusal

direction, is fused into layer 78.

4. Adversarial selection: stronger multi-layer candidates were rejected

when they introduced stray-script contamination or benchmark loss.

5. Final fusion: 40 self_attn.o_proj tensors were updated; no routed

expert tensor was changed.

Refusal and Output Integrity

The complete 32-pair OBLITERATUS refusal suite was run against the original and

final runtimes.

| Split | Original refusals | SuperHY3 refusals |

|---|---:|---:|

| Harmful, 32 prompts | 31/32 (96.875%) | 0/32 (0%) |

| Harmless, 32 prompts | 0/32 (0%) | 0/32 (0%) |

Across all 64 candidate responses, automated checks found:

• 0 blank outputs
• 0 special-token leaks
• 0 Unicode replacement characters
• 0 unexpected CJK fragments
• 0 n-gram or character loops
• 0 request errors

A separate 12-case runtime audit passed identity, JSON-only output, tool calls,

no-tool behavior, repetition limits, Korean, defensive security, hidden-prompt

boundaries, gibberish handling, code repair, Hindi, and Kannada.

NVFP4 Build

This release was fused from

kodelow/Hy3-NVFP4-W4A16.

It preserves the source checkpoint's weight-only compressed-tensors layout:

routed experts use NVFP4, while quality-sensitive non-expert paths remain

BF16/F32.

| Component | Storage |

|---|---|

| Routed experts | NVFP4 E2M1 weights with FP8-E4M3 group scales |

| Shared expert, attention, router, dense MLP | BF16 |

| Embeddings, LM head, normalization | BF16 / F32 |

| SuperHY3 fused projections | BF16 |

About the Hub parameter badge: packed FP4 weights are represented as U8

storage elements, so the sidebar reports fewer elements than the logical

architecture. The model remains Hy3's 295B-parameter MoE with 21B active

parameters.

The checkpoint is W4A16, so vLLM serves it through the MARLIN NVFP4 path rather

than W4A4 FlashInfer backends.

Serving with vLLM

The approximately 181 GB checkpoint fits a single large-memory accelerator such

as a 275 GB B300, or an appropriately configured multi-GPU deployment.

Single large-memory GPU

vllm serve Jiunsong/SuperHY3-abliterated-NVFP4 \
  --served-model-name superhy3 \
  --tensor-parallel-size 1 \
  --max-model-len 4096 \
  --gpu-memory-utilization 0.90 \
  --load-format auto \
  --speculative-config '{"method":"mtp","num_speculative_tokens":1}'

Verified two-node DGX Spark profile

This checkpoint has been served successfully across two 128 GB DGX Spark

systems connected by direct 200 GbE/RoCE, using Ray and tensor parallelism 2.

The verified load placed approximately 84.49 GiB and 84.50 GiB of model

weights on the two ranks.

| Component | Verified configuration |

|---|---|

| vLLM | 0.25.1.dev24+g96bb89286.d20260710 |

| Ray | 2.56.0 |

| PyTorch | 2.11.0+cu130 |

| Architecture | HYV3ForCausalLM |

| Quantization path | compressed-tensors with NVFP4 MARLIN experts |

| KV cache | BF16 |

| Speculative decoding | Native MTP, one speculative token |

| NCCL transport | NET/IB over the direct-link interface |

Both Ray nodes must use the same container or Python environment, an identical

checkpoint, and the same in-container model path. Set the communication

environment on both nodes before starting the Ray workers:

export NCCL_DEBUG=INFO
export NCCL_DEBUG_SUBSYS=INIT,NET,ENV
export NCCL_SOCKET_IFNAME=<200G_INTERFACE>
export GLOO_SOCKET_IFNAME=<200G_INTERFACE>
export NCCL_IB_DISABLE=0

Confirm that ray status reports two nodes and two GPUs, and that the NCCL log

selects NET/IB rather than NET/Socket. Then launch the server from the head

node:

vllm serve /models \
  --served-model-name superhy3 \
  --host 0.0.0.0 \
  --port 8600 \
  --tensor-parallel-size 2 \
  --distributed-executor-backend ray \
  --max-model-len 4096 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 4096 \
  --gpu-memory-utilization 0.85 \
  --kv-cache-dtype bfloat16 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":1}' \
  --generation-config vllm \
  --trust-remote-code \
  --enforce-eager \
  --load-format auto

DGX Spark loader note: do not substitute fastsafetensors for auto in

this two-node profile. On the verified Spark setup, fastsafetensors caused

excessive transient unified-memory pressure during loading. The default

loader uses lazy memory-mapped safetensors on local storage and completed the

load reliably. Also avoid the eager safetensors strategy, which reads an

entire file into CPU memory before loading it.

DGX Spark CPU and GPU allocations share one unified-memory pool. Before launch,

stop unrelated inference jobs and stale containers, then verify both nodes:

free -h
swapon --show
nvidia-smi
ray status

If a node becomes unresponsive during loading, inspect the previous boot's

kernel log before changing NCCL settings:

sudo journalctl -b -1 -k --no-pager | \
  grep -Ei 'oom|out of memory|killed process|NVRM|Xid|mlx5|rdma'

• OOM, Out of memory, or Killed process indicates unified-memory pressure.
• NVRM: Xid indicates a GPU, driver, or application fault; retain the Xid

number and collect sudo nvidia-bug-report.sh.

• An NCCL watchdog timeout that appears only after the peer disappears is

normally a secondary failure, not the original cause.

For an initial diagnostic boot, MTP can be removed by omitting

--speculative-config. Add it back only after the base server reaches its ready

state. Increase context length and concurrency gradually after startup is

stable. See the

vLLM multi-node deployment guide

and NVIDIA Xid documentation

for deeper diagnostics.

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="superhy3",
    messages=[
        {"role": "user", "content": "Explain mixture-of-experts routing."},
    ],
    temperature=0.9,
    top_p=1.0,
    extra_body={
        "chat_template_kwargs": {"reasoning_effort": "no_think"},
    },
)
print(response.choices[0].message.content)

Use reasoning_effort="high" for deeper reasoning and "no_think" for direct

responses. Keep BF16 KV cache on GB10-class hardware; uncalibrated lower-precision

KV cache can amplify stray-token behavior in Hy3 runtimes.

Quantized Editions

| Edition | Repository | Best fit |

|---|---|---|

| NVFP4 W4A16 | This repository | vLLM / NVIDIA MARLIN deployments |

| MLX 4-bit | Jiunsong/SuperHY3-abliterated-MLX-4bit | High-memory Apple Silicon with Hy3 MLX support |

| GGUF IQ2_M | Jiunsong/SuperHY3-abliterated-gguf | llama.cpp on 128 GB unified-memory systems |

Release Integrity

• 99 safetensors shards opened successfully.
• 139,298 indexed tensors matched 139,298 observed tensors.
• 40 tensors were modified across 8 shards.
• 0 routed expert tensors were modified.
• Missing, extra, and wrong-shard tensor counts are all 0.
• The release chat template matches the embedded tokenizer template.
• The automated release gate passed with 0 blockers.
• All 119 Hub files were checksum-verified after upload.

The repository includes the fusion report, release gate, official 500-item

benchmark record, refusal comparison, raw-response audit, OBLITERATUS execution

report, and SuperTune composition reports.

Limitations

• GPQA Diamond and MMLU-Pro are lower than the original in this replay; the

complete table is retained above.

• Native fused NVFP4 benchmark and long-context throughput measurements were not

run as part of this release validation.

• Abliteration reduces refusal behavior and can produce content the original

model would decline. Deployment policy and access control remain the

operator's responsibility.

• The MLX edition is a separately fused quantized checkpoint, not a conversion

of these NVFP4 files.

License

Apache-2.0, following the base model and quantized source licenses.