K2-Horizon-32B Heretic

BF16 weights of a refusal-attenuated derivative of the IFM/K2-Horizon-32B model.

The pipeline is: Heretic refusal attenuation (trial 124: 3/100 refusals, KL divergence 0.056) → LoRA merge into the base BF16 weights. This release is the merged full-precision model with no further training.

The modeling files in this repository include a small fix that returns hidden states from the model forward pass (required by Heretic). It does not change inference behavior. trust_remote_code=True is required.

---

Files

model-00001-of-00064.safetensors … model-00064-of-00064.safetensors
config.json / tokenizer.json / chat_template.jinja / modeling_k2_horizon.py …

Quantized GGUF versions (BF16/Q8_0/Q6_K/Q4_K_M/Q2_K) are available at soyaakinohara/K2-Horizon-32B-heretic-gguf.

---

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "soyaakinohara/K2-Horizon-32B-heretic",
    dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
tok = AutoTokenizer.from_pretrained(
    "soyaakinohara/K2-Horizon-32B-heretic",
    trust_remote_code=True,
)

Full-precision weights need ~70 GiB of memory. For 2x16 GiB VRAM setups, use the Q4_K_M GGUF with a K2-compatible llama.cpp build instead.

---

日本語

概要

IFM/K2-Horizon-32B をベースに、Heretic で無検閲化したモデルの BF16 重みです。 trial 124(拒否 3/100、KL 0.056)をマージしたもので、追加学習はしていません。

同梱の modeling ファイルには hidden states を返す小さな修正が入っています。 Heretic 用のもので、推論の動作は変わりません。 使うときは trust_remote_code=True が必要です。

使い方

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "soyaakinohara/K2-Horizon-32B-heretic",
    dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

フル精度だと約 70GB 要ります。 VRAM が足りないときは GGUF 版を使ってください。