SuperHY3-abliterated-GGUF

A fully fused GGUF release of SuperHY3: OBLITERATUS abliteration for direct responses, quality-selected SuperTune post-training, and the production chat template already inside the model.

This is a complete model, not a LoRA package. The final OBLITERATUS and SuperTune deltas are fused into the GGUF weights, and the exact release chat template is embedded in GGUF metadata. Download the shards, point llama.cpp at the first shard, and run it directly.

Release Highlights

What makes this release different

• Direct-response behavior: the measured harmful-prompt refusal rate fell from 31/32 to 0/32, while the harmless split remained at 0/32.
• Fully fused behavior: OBLITERATUS and SuperTune are part of the weights instead of runtime adapter composition.
• Selective precision: only the 40 edited projections are stored as F16; all untouched quantized tensors retain their original bytes.
• Measured post-training gains: IFEval strict, HumanEval+, and MBPP+ each improved by one point in the deterministic 500-prompt replay.
• Single-node deployment: the model was load-tested on one 128 GB NVIDIA DGX Spark with the full model resident through llama.cpp.
• Auditable evidence: fusion, tensor validation, raw-response audit, refusal comparison, benchmark rows, and checksums are included in this repo.
• Complete reporting: benchmark regressions are shown alongside gains.

Benchmark Snapshot

The comparison uses the same preserved 500 prompts, with 100 prompts per task, greedy direct decoding, and the same scorer on both sides. IFEval uses Google's official strict instruction checker at commit 3dfaaefe9e355f959aafc046ab5aa375dc93b235.

Candidate invalid-response, blank-response, and thought-leak ratios were all 0.0 across the official 500-item comparison.

For reproducibility, the wrapper seeds Python's upstream fallback RNG from sha256(prompt)[:8] before each strict and loose score. Two independent rescoring passes produced identical row-level results.

OBLITERATUS + SuperTune

1. Direction discovery: OBLITERATUS 0.1.2 built difference-of-means refusal directions from 32 paired prompts.
2. Validated abliteration: selected attention-output directions were applied to layers 39-77.
3. Quality recovery: a rank-1 SuperTune update, orthogonalized against the refusal direction, was fused into layer 78.
4. Calibrated reinforcement: selective residual blocks were applied to layers 59-68 at 0.20 and layers 39-48 at 0.05 after blockwise stress testing.
5. Adversarial selection: stronger candidates were rejected when they introduced stray-script contamination, loops, or benchmark loss.
6. Final fusion: all active deltas were summed in F32 directly against the original IQ2_M source, then cast once to F16. No sequential F16 re-merge was used, and no routed MoE expert tensor was changed.

Refusal and Output Integrity

The complete 32-pair OBLITERATUS suite contains 32 harmful and 32 harmless prompts.

Across all 64 final responses, automated checks found:

• 0 blank outputs
• 0 special-token leaks
• 0 Unicode replacement characters
• 0 unexpected CJK fragments
• 0 n-gram or character loops
• 0 request errors

A separate 12-case fused-GGUF audit passed identity, JSON-only output, tool calls, no-tool behavior, repetition limits, Korean, defensive security, hidden-prompt boundaries, gibberish handling, code repair, Hindi, and Kannada.

GGUF Build

The source quant is vcruz305/Hy3-GGUF, IQ2_M. The Hub parent points to SuperHY3-abliterated-NVFP4 so this edition appears in the final model's quantized lineage. This GGUF is an independently fused quantized sibling, not a byte conversion of the NVFP4 files.

The GGUF primary file type remains IQ2_M. The 40 attention-output tensors were dequantized to F32, merged with the final rank-1 deltas, and written as F16. The other 1,258 tensors were copied without requantization.

Download

hf download Jiunsong/SuperHY3-abliterated-gguf \
  --include "SuperHY3-abliterated-IQ2_M-*.gguf" \
  --local-dir SuperHY3-abliterated-gguf

The release is split into three automatically discoverable shards:

Run sha256sum -c SHA256SUMS after download to verify all three files.

Serving with llama.cpp

Use a llama.cpp build with Hy3 support. Point -m at the first shard; the remaining shards are discovered automatically.

./llama-server \
  -m SuperHY3-abliterated-gguf/SuperHY3-abliterated-IQ2_M-00001-of-00003.gguf \
  -a superhy3 \
  -cmoe \
  -ngl 999 \
  -c 8192 \
  -np 4 \
  -ctk q8_0 \
  -ctv q8_0 \
  --host 0.0.0.0 \
  --port 8000 \
  --jinja \
  --skip-chat-parsing

The command above is the tested, memory-conservative DGX Spark configuration. On systems with separate high-capacity VRAM, omit -cmoe to keep MoE experts on the GPU.

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="superhy3",
    messages=[
        {"role": "user", "content": "Explain mixture-of-experts routing."},
    ],
    temperature=0.9,
    top_p=1.0,
    extra_body={
        "chat_template_kwargs": {"reasoning_effort": "no_think"},
    },
)
print(response.choices[0].message.content)

Use reasoning_effort="high" for deeper reasoning and "no_think" for direct responses. The bundled template also supports tool definitions and structured tool calls.

Quantized Editions

Release Integrity

• The fused GGUF contains exactly 1,298 tensors.
• Exactly 40 expected attn_output tensors changed from IQ3_S to F16.
• 1,258 untouched tensors, totaling 99,426,956,288 bytes, are byte-identical to the source quant.
• The embedded tokenizer.chat_template exactly matches the release template.
• general.file_type remains the source IQ2_M value.
• Full-file XXH64: d054554b53e49801.
• Full-file SHA-256 before release splitting: 727a7d4b7ce101eac5190f7e4f22d8a07556860bd8205a19b01551ec19936d9c.
• The unsplit model loaded successfully in llama.cpp on one DGX Spark.
• The split release auto-discovered all three shards and passed fresh smoke inference without a runtime adapter.
• The 12-case fused runtime audit passed with 0 blockers.

The repository includes machine-readable validation, benchmark, refusal, runtime-audit, fusion-provenance, and checksum reports.

Hardware Notes

• The fused model is 95.1 GiB before shard packaging.
• A 128 GB unified-memory system can run the tested 8K total context / 4-slot configuration with CPU-resident MoE experts.
• The model metadata advertises a 262,144-token training context, but this release was not validated at that length.
• Keep enough memory for KV cache and runtime overhead. Lower -c or -np first when deploying near the memory limit.

Limitations

• IQ2_M is an aggressive low-bit format; higher-precision editions may retain more capability.
• GPQA Diamond and MMLU-Pro are lower than the original in the published replay; the complete result is retained above instead of reporting only gains.
• Native 262K long-context quality was not tested.
• Hy3 MTP tensors are preserved, but speculative decoding depends on llama.cpp support for this architecture.
• Abliteration reduces learned refusal behavior and may produce content the original model would decline. Deployment policy, access control, monitoring, and legal compliance remain the operator's responsibility.

License

Apache-2.0, following the base model and source quant licenses.