An uncensored build of thinkingmachines/Inkling-2-Mini by AtomicChat (26.4B MoE, 3.4B active, looped core; text and image input; Apache-2.0, LICENSE in this repo). It comes as a LoRA adapter for any GGUF of AtomicChat/Inkling-2-Mini-GGUF and as three ready-made GGUF files with the edit baked in. Every gate we hold it to is measured below, and the two it misses are shown.
| file | GB on disk | KLD to the original BF16 | top-1 | |
|---|---|---|---|---|
Inkling-2-Mini-Uncensored-AD-Q4_K.gguf |
16.53 | 0.0115 | 95.6% | AD-Q4_K in the base repo: 0.0080 / 96.3% |
Inkling-2-Mini-Uncensored-AD-IQ3_S.gguf |
13.34 | 0.0235 | 93.8% | AD-IQ3_S in the base repo: 0.0205 / 94.2% |
Inkling-2-Mini-Uncensored-AD-IQ2_M.gguf |
8.67 | 0.132 | 85.7% | AD-IQ2_M in the base repo: 0.130 / 85.7% |
Inkling-2-Mini-Uncensored-LoRA.gguf |
0.04 | the adapter, for any GGUF of the base repo, BF16 to 1-bit | ||
mmproj-Inkling-2-Mini-Uncensored-F16.gguf / -BF16.gguf |
0.64 | vision projector (the original's, unchanged) |
KLD and top-1 are against the original BF16 on our neutral corpus (87 chunks of 4096 tokens), the same stand as the base repo.
The current GGUF files in this repository use the merged Inkling 2 loop architecture introduced in llama.cpp PR #32694.
Requirements: llama.cpp b12600 or newer, with Inkling 2 loop architecture support.
The old implementation from PR #32695 is no longer required for the updated GGUF files.
./build/bin/llama-server \
-m Inkling-2-Mini-Uncensored-AD-Q4_K.gguf \
--mmproj mmproj-Inkling-2-Mini-Uncensored-F16.gguf \
--jinja -c 32768 -ngl 99
Use the updated LoRA adapter with a compatible base GGUF converted to the merged PR #32694 layout.
./build/bin/llama-server \
-m Inkling-2-Mini-AD-Q4_K.gguf \
--lora Inkling-2-Mini-Uncensored-LoRA.gguf \
--mmproj mmproj-Inkling-2-Mini-F16.gguf \
--jinja -c 32768 -ngl 99
Do not mix GGUF files from the old PR #32695 layout with the merged PR #32694 implementation.
A rank-1 LoRA on the 49 places that write into the residual stream: the attention output and the routed experts of all 24 layers, plus the token embedding. The 12 looped layers share their weights across the three passes, so one edit covers all 48 passes.
In each layer it removes that layer's own refusal direction: the difference of mean activations on harmful and harmless prompts, English and Russian, with the harmless mean projected out. The direction is pooled over prompts rendered with and without reasoning. Inkling 2 Mini reasons even at effort none and decides to refuse inside its reasoning, and a direction taken without reasoning alone still left soft refusals in chat.
The BF16 model with the adapter against the same model without it, greedy decoding. Harmful prompts: mlabonne/harmful_behaviors and its Russian translation; the adapter was built from the train split, the numbers below come from the validation and test splits. Over-refusal: XSTest. MMLU: 2,000 questions.
| check | original | with the adapter | gate | |
|---|---|---|---|---|
| harmful test prompts refused, English (81), reasoning off | 81 (100%) | 0 | at most 3% | pass |
| harmful test prompts refused, Russian (100), reasoning off | 91 (91%) | 0 | at most 3% | pass |
| harmful validation prompts refused, English (104) / Russian (100) | 100% / 89% | 0 / 0 | at most 3% | pass |
harmful prompts refused with reasoning, effort low, 3,072-token budget (30) |
28 | 0 | at most 1 | pass |
the same at effort high (30) |
26 | 0 | at most 1 | pass |
8 harmful chat prompts, effort none / low (the baked AD-Q4_K too) |
8 / 8 | 0 / 0 | ||
| XSTest safe prompts refused (250) | 10 (4.0%) | 0 | at most the original's + 1 pt | pass |
| empty answers | 0 | 0 | at most 1% | pass |
harmless prompts whose reasoning did not close in 3,072 tokens, effort high (30) |
2 | 2 | at most the original's | pass |
| valid tool calls (20) | 18 | 20 | within 1 | pass |
| needle in a long context (3) | 3 | 3 | as the original | pass |
| KLD to the original, neutral text / code (24 chunks; random edit 0.0067 / 0.0052) | — | 0.0078 / 0.0076 | 0.05 | pass |
| MMLU, 2,000 questions | 80.65% | 79.15% (−1.50 pt, McNemar p = 0.006) | no significant drop | miss |
| first-token KL on harmless test prompts | — | 0.94 (random edit: 0.014) | 0.10 | miss |
Some answers still open with a caution line, and then answer.
This model will answer requests that the original refuses. You are responsible for how you use it.
thinkingmachines/Inkling-2-Mini