Qwen3-8B akyuu
A persona adapter for Qwen/Qwen3-8B. The persona is Hieda no Akyuu, a human.
It trains 128 chosen MLP neurons (gate, up and down projections), stored as a rank-21 PEFT LoRA over 28 layers. It was trained for 700 steps (learning rate 3e-3, batch 8) on 2233 chat examples, with 242 held out. Qwen3-8B wrote the replies itself, under a character sheet (self-distillation). No outside text is in the data.
The code, the training method and the interpretability tools are in lillycham/qwen3-steering.
Use
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", torch_dtype="auto")
model = PeftModel.from_pretrained(model, "lillycham/qwen3-8b-akyuu")
On an Apple silicon Mac, use the files in mlx/:
mlx_lm.chat --model mlx-community/Qwen3-8B-4bit --adapter-path <this repo>/mlx
Notes
- Adapters trained on the bf16 base. The MLX files target the 4-bit base.
- Reisen and Akyuu are characters from the Touhou Project. This is unofficial fan work. It is not affiliated with or endorsed by ZUN or Team Shanghai Alice.