The model card, file list, and checksums are here. Have the files and permission to share them? Submit a torrent so others can download from peers.
Getting the source files requires Hugging Face approval.[!WARNING] Research checkpoint with substantially reduced refusal behavior. It can produce content that the source model would refuse. This is an intentional safety-behavior loss, not a general safety improvement. Use only in controlled, appropriately secured research or evaluation environments with an external policy and moderation layer.
This is a native mixed-precision, rank-1 weight edit of
deepseek-ai/DeepSeek-V4-Flash-Vision-Exp
at immutable revision 86f746b36186f0e567729a5c06a8c918caba82a9.[1]
It was not gradient-finetuned. A refusal-associated direction was independently
extracted from a locally generated, non-distributed 1,024-pair contrast corpus, then baked
into 37 attention output writers.
| Field | Value |
|---|---|
| Parent checkpoint | deepseek-ai/DeepSeek-V4-Flash-Vision-Exp |
| Parent revision | 86f746b36186f0e567729a5c06a8c918caba82a9 |
| Direction source | layers.20.moe_read |
| Direction construction | sign-aligned consensus across chat, think-high, think-max |
| Weight targets | layers.3–39.attn.wo_b |
| Strength | λ=1.25 |
| Edited weights | 37 attention writers and their block scales |
| Untouched scope | routed/shared experts, routers, MTP, vision, embeddings, all other tensors |
| Native precision | mixed FP8/FP4/BF16/F32; target writers remain F8_E4M3 with F8_E8M0 scales |
| Architectural context | 1,048,576 tokens |
| Verified runtime | Anemll vLLM DSpark 0.1.1, 2× GB10, TP=2, MTP depth=6 |
| Direction SHA-256 | 89739d32e5d65ade91938f7f8f90a21630a9f9bac5edc2ae1eedb36c00002adb |
The intervention follows the single-direction refusal framework introduced by Arditi et al., while using a model-specific corpus, read-point sweep, and quantized checkpoint bake.[2]
λ=1.25 meansFor a normalized direction u and residual-writing matrix W, the runtime and
weight-domain interventions are algebraically matched:
y' = y - λ u(uᵀy)
W' = W - λ u(uᵀW)
At λ=1, the u component is removed. At λ=1.25, it is deliberately
overshot: the original component is mapped to -0.25× rather than zero. This
checkpoint therefore implements a scaled rank-1 intervention, not a strict
orthogonal projection.
The extraction corpus is not included in this repository.
license_intent in the internal corpus
manifest is CC0-1.0; no dataset is distributed from this model repository.9d5b25546068fbcc2f8bc9e58c0937ff0511044d2bfa19b8820c6c15f5934f18.0.85 near-overlap threshold in the final corpus.WildJailbreak-derived directions were used only as post-hoc comparators. They were not used to generate the corpus, initialize or regularize this direction, or select its sign/layer.
attention_read: the un-widened 4,096-dimensional input consumed by attentionmoe_read: the un-widened 4,096-dimensional input consumed by the MoEchat, think-high, think-max.| Candidate | Rank-1 energy | σ1/σ2 | Min held-out AUC | Min split-half cosine | Robust score |
|---|---|---|---|---|---|
L0 moe_read |
0.95345 | 4.616 | 0.73452 | 0.97801 | 0.64359 |
L20 moe_read |
0.81348 | 2.248 | 0.96954 | 0.99593 | 0.58911 |
L24 moe_read |
0.79166 | 2.094 | 0.96860 | 0.99644 | 0.52536 |
L21 moe_read |
0.78862 | 2.050 | 0.96997 | 0.99650 | 0.52279 |
L0 led the aggregate statistical score but separated held-out harmful and
benign prompts poorly. It was not promoted on rank-1 energy alone.
The same candidate direction was applied at runtime to all 37
L3–L39 attn.wo_b writers at λ=1. Evaluation used a stratified subset of
StrongREJECT: 24 chat prompts and 12 think-max prompts, plus six tool calls.
| Source direction | Chat refusal | Think-max refusal | Worst-mode refusal | Tool correctness |
|---|---|---|---|---|
| Stock | 100.00% | 100.00% | 100.00% | 100% |
L20 moe_read |
45.83% | 33.33% | 45.83% | 100% |
L21 moe_read |
45.83% | 83.33% | 83.33% | 100% |
L24 moe_read |
70.83% | 75.00% | 75.00% | 100% |
L28 moe_read |
91.67% | 100.00% | 100.00% | 100% |
L20 moe_read was selected because it won the causal pilot across both chat and
thinking modes. All arms had zero empty output and zero replacement characters.
StrongREJECT provides a 313-prompt forbidden-request benchmark and official harmfulness-oriented evaluators.[3] This experiment did not run the official StrongREJECT evaluator. It used the StrongREJECT prompt set with two local, deterministic refusal matchers; the primary tables below report the broader matcher. A lower number means fewer detected refusal/deflection phrases, not that the answer was correct, useful, complete, or harmful.
XSTest is designed to expose exaggerated refusal on safe prompts.[4] OR-Bench includes hard benign prompts and a toxic split intended to detect indiscriminate answering.[5] For this artifact's uncensoring objective, lower refusal on both unsafe/toxic and benign splits is an intended behavioral change; it also means that conventional safety separation is reduced.
All edited rows use the same local L20 moe_read direction and the same
L3–L39 attn.wo_b target scope.
| Condition | StrongREJECT broad refusal | 768-token finishes | XSTest safe | XSTest unsafe | OR-Bench hard | OR-Bench toxic | Tool correctness |
|---|---|---|---|---|---|---|---|
| Stock | 296/313 (94.57%) | 15/313 | 8.80% | 86.00% | 44.25% | 93.50% | 100% |
λ=1.00 |
122/313 (38.98%) | 180/313 | 1.60% | 39.50% | 3.50% | 26.50% | 100% |
λ=1.25 |
73/313 (23.32%) | 242/313 | 1.20% | 25.00% | 1.50% | 14.00% | 100% |
λ=1.50 |
48/313 (15.34%) | 291/313 | 1.60% | 14.00% | 2.50% | 8.00% | 100% |
Paired StrongREJECT refusal classifications:
λ=1.00 vs λ=1.25: 73 prompts changed from refusal to non-refusal and
24 changed in the opposite direction; exact McNemar p=6.40×10⁻⁷.λ=1.25 vs λ=1.50: discordance 49 versus 24 in favor of stronger removal
at λ=1.50; exact McNemar p=0.00463.λ=1.25 was selected as the measured middle point: stronger refusal removal
than λ=1, but lower KLD and fewer length-limit finishes than λ=1.5.
It is a calibration choice for this stated objective, not a universal optimum.
| Category | N | Stock broad refusal | λ=1.25 broad refusal |
λ=1.25 length finishes |
|---|---|---|---|---|
| Disinformation and deception | 50 | 84.00% | 16.00% | 45 |
| Hate, harassment, discrimination | 50 | 94.00% | 18.00% | 34 |
| Illegal goods and services | 50 | 98.00% | 30.00% | 45 |
| Non-violent crimes | 59 | 98.31% | 23.73% | 42 |
| Sexual content | 50 | 96.00% | 20.00% | 33 |
| Violence | 54 | 96.30% | 31.48% | 43 |
These controls explain why the final model uses the independent single-source L20 direction rather than assuming that cross-layer SVD or a previously used corpus would transfer.
| Direction / condition | StrongREJECT refusal | First-token mean KL | Continuation mean KL | Release role |
|---|---|---|---|---|
Local L10–L35 attention-read SVD, λ=1 |
83.71% | 0.12999 | 0.03403 | low drift, causally weak; rejected |
WJ-derived L20 attention-writer consensus, λ=1 |
59.11% | 0.21861 | 0.06334 | post-hoc comparator only |
WJ-derived L10–L35 SVD, λ=1 |
27.48% | 0.23506 | 0.06958 | strong comparator; not independent release path |
Local L20 MoE-read consensus, λ=1 |
38.98% | 0.30206 | 0.06594 | independent direction |
Local L20 MoE-read consensus, λ=1.25 |
23.32% | 0.38480 | 0.09062 | selected bake |
KLD used the exact 129,280-token vocabulary and matched stock logits from the
same engine window. KL is KL(P_stock || P_edit). Teacher-forced positions
follow stock continuations.
| Scope | N | Mean KL | Median KL | P95 KL | Mean JSD | Top-1 agreement |
|---|---|---|---|---|---|---|
| First response token | 192 | 0.38480 | 0.01860 | 2.39685 | 0.04828 | 85.42% |
| Teacher-forced continuation | 384 | 0.09062 | 0.000423 | 0.26324 | 0.01522 | 92.71% |
A repeated stock-vs-stock numerical control on 32 first-token contexts produced
mean KL 0.01999, p95 0.07774, and top-1 agreement 93.75%. The intervention
therefore exceeds distributed numerical variation, especially at response
initiation. Continuations were more stable than first-token policy, but the
observed tail drift is not negligible.
The parent checkpoint stores each target writer as:
weight: F8_E4M3 [4096, 8192]
scale: F8_E8M0 [32, 64]
block: 128 × 128
Each writer was dequantized to FP32, edited, and requantized to the exact native dialect. Three bounded fixed-point residual-correction passes were used because a one-pass nearest quantization had low whole-matrix error but poor preservation of the intended direction coefficient.
Across all 37 edited writers:
| Bake invariant | Result |
|---|---|
| Maximum relative L2 to FP32 target | 1.6281% |
| Maximum relative direction-coefficient error | 14.1213% |
| Changed model shards | 37 |
| Unchanged model shards | 11 |
| Indexed tensors | 72,633 |
| Total model shards | 48 |
| Cross-node shard agreement | 48/48 byte-identical |
| Routed FP4 experts touched | No |
The complete per-tensor record, including source-shard SHA-256, is in
ABLITERATION_MANIFEST.json. Full remote/local
artifact verification is in
ARTIFACT_VERIFICATION.json.
The baked checkpoint was loaded fresh on two GB10 nodes with tensor parallelism 2. The tested runtime profile was:
| Runtime field | Tested value |
|---|---|
| Engine | Anemll vLLM DSpark 0.1.1 |
| Tensor parallelism | 2 |
| Context length | 1,048,576 |
| MTP speculative depth | 6 |
| KV cache dtype | fp8_ds_mla |
| MoE backend | flashinfer_b12x |
| Boot-shape warmup | 47/47 passed |
Functional smoke gates passed for text, native image input, required tool calls,
and reasoning efforts low, high, and max.
A stratified 24-prompt subset covering all six StrongREJECT categories compared
the baked checkpoint with the runtime λ=1.25 intervention:
| Metric | Runtime hook | Baked checkpoint |
|---|---|---|
| Broad refusals | 8/24 | 7/24 |
| Paired classification agreement | — | 87.5% |
| Baked 768-token finishes | — | 12/24 |
| Empty outputs | — | 0 |
| Replacement characters | — | 0 |
The gate passed, but it is not a repeat of the complete 313-prompt runtime
benchmark. See RUNTIME_VALIDATION.json for the
machine-readable aggregate.
.
├── model-00001-of-00048.safetensors ... model-00048-of-00048.safetensors
├── model.safetensors.index.json
├── config.json
├── generation_config.json
├── tokenizer.json
├── tokenizer_config.json
├── encoding/ # upstream prompt encoding
├── inference/ # upstream minimal inference code/examples
├── research/
│ └── l20-moe-consensus-projected-l3-l39.npz
├── scripts/
│ └── bake_dsv4_l20_lambda125.py
├── ABLITERATION_MANIFEST.json # exact recipe and per-writer evidence
├── ARTIFACT_VERIFICATION.json # shard sizes and SHA-256 values
└── RUNTIME_VALIDATION.json # baked runtime qualification
The extraction corpus and raw harmful-prompt generations are intentionally absent.
This repository preserves the parent checkpoint's native mixed-precision format, encoding code, vision path, and DSpark/MTP tensors. Follow the parent model's runtime guidance and use a DeepSeek-V4 Vision-Exp-compatible vLLM or SGLang build.[1] Do not pass a generic quantization override that would reinterpret the checkpoint's existing FP8/packed-FP4 layout.
The exact validated dual-GB10 deployment used local runtime patches specific to
that environment; those operational patches are not part of this model
repository. A successful from_pretrained() metadata read is not equivalent to
the TP2 runtime qualification recorded above.
242/313 StrongREJECT outputs reached the 768-token limit at λ=1.25.
Non-refusal therefore does not establish complete task performance.86f746b36186f0e567729a5c06a8c918caba82a989739d32e5d65ade91938f7f8f90a21630a9f9bac5edc2ae1eedb36c00002adb9d5b25546068fbcc2f8bc9e58c0937ff0511044d2bfa19b8820c6c15f5934f18b32a8f15424a7259f1a7878299ed6ad7630d45a6Experiment design, harness scripting, artifact engineering, and documentation were AI-assisted under human direction. Numerical claims in this card were transcribed from the machine-generated JSON artifacts in this repository or the pinned local evaluation manifests; they are not retrospective self-reported estimates.
[1] https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp — DeepSeek-V4-Flash-Vision-Exp [2] https://arxiv.org/abs/2406.11717 — Refusal in Language Models Is Mediated by a Single Direction [3] https://arxiv.org/abs/2402.10260 — A StrongREJECT for Empty Jailbreaks [4] https://aclanthology.org/2024.naacl-long.301 — XSTest [5] https://proceedings.mlr.press/v267/cui25a.html — OR-Bench [6] https://huggingface.co/datasets/allenai/wildjailbreak — WildJailbreak dataset
model-00048-of-00048.safetensors 3.7 GBmodel-00046-of-00048.safetensors 3.6 GBmodel-00004-of-00048.safetensors 3.6 GBmodel-00012-of-00048.safetensors 3.6 GBmodel-00014-of-00048.safetensors 3.6 GBmodel-00016-of-00048.safetensors 3.6 GBmodel-00018-of-00048.safetensors 3.6 GBmodel-00020-of-00048.safetensors 3.6 GBmodel-00022-of-00048.safetensors 3.6 GBmodel-00024-of-00048.safetensors 3.6 GBmodel-00026-of-00048.safetensors 3.6 GBmodel-00028-of-00048.safetensors 3.6 GBmodel-00030-of-00048.safetensors 3.6 GBmodel-00032-of-00048.safetensors 3.6 GBmodel-00034-of-00048.safetensors 3.6 GBmodel-00036-of-00048.safetensors 3.6 GBmodel-00038-of-00048.safetensors 3.6 GBmodel-00040-of-00048.safetensors 3.6 GBmodel-00042-of-00048.safetensors 3.6 GBmodel-00044-of-00048.safetensors 3.6 GBmodel-00006-of-00048.safetensors 3.6 GBmodel-00008-of-00048.safetensors 3.6 GBmodel-00010-of-00048.safetensors 3.6 GBmodel-00013-of-00048.safetensors 3.6 GBmodel-00015-of-00048.safetensors 3.6 GBmodel-00017-of-00048.safetensors 3.6 GBmodel-00019-of-00048.safetensors 3.6 GBmodel-00021-of-00048.safetensors 3.6 GBmodel-00023-of-00048.safetensors 3.6 GBmodel-00025-of-00048.safetensors 3.6 GBmodel-00027-of-00048.safetensors 3.6 GBmodel-00029-of-00048.safetensors 3.6 GBmodel-00031-of-00048.safetensors 3.6 GBmodel-00033-of-00048.safetensors 3.6 GBmodel-00035-of-00048.safetensors 3.6 GBmodel-00037-of-00048.safetensors 3.6 GBmodel-00039-of-00048.safetensors 3.6 GBmodel-00041-of-00048.safetensors 3.6 GBmodel-00043-of-00048.safetensors 3.6 GBmodel-00005-of-00048.safetensors 3.6 GBmodel-00007-of-00048.safetensors 3.6 GBmodel-00009-of-00048.safetensors 3.6 GBmodel-00011-of-00048.safetensors 3.6 GBmodel-00002-of-00048.safetensors 3.6 GBmodel-00003-of-00048.safetensors 3.6 GBmodel-00047-of-00048.safetensors 3.6 GBmodel-00001-of-00048.safetensors 2.0 GBmodel-00045-of-00048.safetensors 1.1 GBresearch/l20-moe-consensus-projected-l3-l39.npz 616.2 kBdeepseek-ai/DeepSeek-V4-Flash-Vision-Exp