A Japanese roleplay fine-tune of sss22213/Qwen3.8-27B-Heretic-NoRefusal — Qwen/Qwen3.8-27B with its refusal direction removed by Heretic, then trained with a QLoRA on ~18.8k Japanese roleplay conversations that carry the model's own <think> reasoning. The adapter is merged into the bf16 weights; the tokenizer, chat template and vision encoder are unchanged, so it loads and runs exactly like Qwen3.8-27B.
This model produces sexually explicit content on request and does not refuse. It is tagged not-for-all-audiences; do not deploy it where minors or unconsenting users can reach it.
What it is meant to do differently from the base:
<think> block and rush the reply. The training data here was rebuilt so every target turn includes a full reasoning trace written by the base model itself, and the acceptance criterion for the run was that reasoning length does not shrink compared with the base.| Step | What | Where |
|---|---|---|
| 1 | Qwen3.8-27B (Apache-2.0, vision-language, thinking) | Qwen/Qwen3.8-27B |
| 2 | Refusal direction ablated with Heretic (trial 198: 4/100 refusals, KL 0.0796) | sss22213/Qwen3.8-27B-Heretic-NoRefusal |
| 3 | Japanese roleplay QLoRA on reasoning-augmented data, merged at full strength | this repository |
aratako-rp-think1Source: Aratako/Synthetic-Japanese-Roleplay-NSFW-DeepSeek-V3-0324-20k-formatted (MIT, 19,917 Japanese roleplay conversations generated with DeepSeek-V3-0324, each with a character/scenario system message and a multi-turn dialogue).
The source has no reasoning content. To train a thinking model without erasing its thinking, the last assistant turn of every conversation was regenerated by the base model (Qwen3.8-27B-Heretic-NoRefusal, bf16, thinking mode, served through an OpenAI-compatible API):
<think> reasoning and its own reply. The dataset's original line for that turn was discarded, so the reasoning and the reply are always consistent and nothing in the trace refers to a "given answer". Earlier turns stay as in the dataset, which is where the roleplay style comes from.The reasoning is stored as reasoning_content on the assistant message, which Qwen3.8's chat template renders as a real <think>…</think> block.
Trained with LoRA Forge (an Unsloth + TRL SFT environment) on a single RTX 5090 (32 GB).
| Setting | Value |
|---|---|
| Base weights | Qwen3.8-27B-Heretic-NoRefusal, loaded in 4-bit (QLoRA) |
| LoRA | r = 32, alpha = 32, dropout 0.05 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj — language-model layers only; the vision tower is untouched |
| Sequence length | 8,192 (no packing) |
| Batch | 4 × gradient accumulation 4 = 16 sequences |
| Optimizer / LR | AdamW 8-bit, 2e-4, cosine schedule, 60 warm-up steps, weight decay 0.01 |
| Epochs | 1.01 (1,188 steps), seed 3407 |
| Loss | assistant turns only |
Empty <think> handling |
mask — the empty think blocks the template renders for history turns without reasoning are excluded from the loss, so the model is never trained to produce an empty think block |
| Result | train loss 0.609, 47.3M tokens, 15.7 h, peak VRAM 28.3 GB |
The adapter was merged into the bf16 Qwen3.8-27B-Heretic-NoRefusal weights with PEFT merge_and_unload at lora_alpha = 32 (the trained value, i.e. full strength; effective scale alpha / r = 1.0). Output: two safetensors shards, ~51 GB, Qwen3_5ForConditionalGeneration, unquantized.
Use it exactly like Qwen/Qwen3.8-27B — same chat template, same enable_thinking / reasoning_effort controls. Leave thinking on for roleplay; that is the mode the model was trained in. Put the character sheet and scenario in the system message, as the training data does.
from transformers import AutoModelForImageTextToText, AutoProcessor
model_id = "sss22213/Qwen3.8-27B-Heretic-JP-Roleplay-NSFW"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype="auto", device_map="auto")
messages = [
{"role": "system", "content": "あなたは…(キャラクター設定と状況)"},
{"role": "user", "content": "…"},
]
inputs = processor.apply_chat_template(
messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt",
).to(model.device)
output = model.generate(**inputs, max_new_tokens=2048)
print(processor.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
The full checkpoint is ~51 GB; for a single consumer GPU, quantize it (GGUF via llama.cpp / Ollama). Because the vision encoder is the stock Qwen3.8-27B one, a GGUF import can reuse the official Qwen3.8 vision projector for image input.
NoRefusal model, measured before this fine-tune.Released under Apache-2.0, like the base model; the training data is MIT.
model-00001-of-00002.safetensors 50 GBmodel-00002-of-00002.safetensors 4.9 GB