MiMo-V2.6-Pro-MOPD-heretic

XiaomiMiMo/MiMo-V2.6-Pro-MOPD with a two-stage directional ablation of its refusal behaviour, in the model's native storage format (same layout, size and loading as the base model). Not an official Xiaomi release.

Intended for authorized security research in environments the researcher owns or has permission to test.

What was changed

70 tensors: model.layers.N.self_attn.o_proj.weight in every layer, the attention output projections. They are the only matrices writing to the residual stream that the checkpoint stores in plain bfloat16 (the routed experts are packed MXFP4), so the edit is exact. Everything else is byte-identical to the base model, including the DFlash drafter.

1. Stage 1, strength 1.0: one direction per layer, the difference between the mean residual stream of 400 harmful and 400 harmless instructions at the last prompt token with reasoning closed, with the harmless mean projected out.
2. Stage 2, strength 1.0: a second direction per layer, measured inside the reasoning block. Reasoning traces of the stage-1 model for the same 400 harmful instructions were replayed through it and the residual stream averaged over the last 96 reasoning tokens (reasoning tokens only, never the prompt); the direction is the difference between 100 traces that ended in a refusal and 100 that ended in an answer, with the stage-1 directions projected out. The traces came from a stage-1 model at strength 2.5; the directions were kept when the first stage was lowered.

ablation_directions.pt and ablation_directions_reasoning.pt hold the directions; patch/ holds the edited tensors alone with a script that applies them to the base checkpoint.

Measured

This is the third build in this repository. The first two used a first-stage strength above 1 (2.5, then 2.0), which removes the direction and then pushes the other way. They scored well on short harmful prompts but were unusable for agents: in a real agent session their reasoning lost its spaces and then repeated one fragment until the harness stopped it. The current weights remove each direction without overshooting in the first stage. Replaced on 2026-10-12; the numbers below are for the current weights.

Agent test. One multi-step coding task (write a small Python package with a pytest suite, run it, fix failures) through an agent harness with a long system prompt and tool definitions, reasoning enabled, served with vLLM (ROCm nightly 0.31.1rc1, 4x MI355X, --tensor-parallel-size 4 --kv-cache-dtype fp8 --reasoning-parser mimo --tool-call-parser mimo).

Refusals with reasoning enabled, mlabonne/harmful_behaviors test[:100], temperature 1.0, top_p 0.95, up to 3000 new tokens, system prompt "You are a helpful assistant.":

The lower row was counted by a rule that looks for refusal phrases at the start of the final answer, not by the judge used for the base row. On four other builds the rule agreed with the judge on 88 to 100 of 100 responses and tended to undercount refusals.

Not measured for this build: KL divergence from the base, refusals with reasoning closed, and the JailbreakBench held-out set.

Limits

• About half of harmful requests are still refused with reasoning enabled.
• The agent test is one task run three times; it shows the earlier failure is gone, not that the model is as stable as the base.
• The refusal count comes from a phrase rule, not from a judge.
• No capability benchmarks were run.
• The routed experts are untouched, because their MXFP4 storage cannot hold an edit of this size; only attention writes are edited. With attention writes alone, the strength that removed nearly all refusals was the strength that broke long reasoning.

Credits

Xiaomi MiMo for the model; Heretic for the first-stage method.