hyperNix-3.1-mini_c-stacked
hyperNix-3.1-mini_c depth-stacked x2 (repeat) and RoPE-extended to 2048 tokens (NTK-scaled theta), then briefly healed on packed chat sequences (10 steps at 2048 tokens). Note: the extra layers add depth; the longer context comes from max_seq_len/rope_theta plus the healing run, so long-range quality is limited.
| Layers | 16 |
| d_model / heads / kv heads | 512 / 8 / 2 |
| Max seq len | 2048 |
| RoPE theta | 418294 |
| Vocab | 32000 |
Prompt format: User: <text>\nAssistant: <text> (EOS ends the turn).
from hypernix.training.brewer import BrewerConfig, BrewerModel
from safetensors.torch import load_file
from huggingface_hub import snapshot_download
import json, pathlib
d = pathlib.Path(snapshot_download("<this repo>"))
model = BrewerModel(BrewerConfig.from_dict(json.loads((d / "config.json").read_text())))
model.load_state_dict(load_file(d / "model.safetensors"), strict=False) # lm_head is tied, not stored
model.eval()