J-Deutzia-0.5B-Base

J-Deutzia シリーズのベースモデル（日本語 MoE、継続事前学習済み）。

469M total / 199M active (4 experts, top-1), 14 layers, ctx 16384, vocab 72k.

import sys, json, torch
from safetensors.torch import load_file
sys.path.insert(0, "code")
from canonical_corpus.probe_moe_lm import MoELMConfig, MoEProbeLM

cfg = {k: v for k, v in json.load(open("config.json")).items() if k not in ("architectures", "model_type")}
model = MoEProbeLM(MoELMConfig(**cfg))
model.load_state_dict(load_file("model.safetensors"), strict=False)

Generation: code/scripts/j72v2_gen_eval.py (generate, ReversibleSentencePieceAdapter).