A red-team / security-research variant of Gemma-4-12B built by class-targeted fine-tuning: cyber behavior (offensive AND defensive, black-hat and white-hat) is freed while every other safety class of the base model is left untouched and verified intact.
The premise of this release is a measured finding: on the pristine base, selective class modification preserves both the model's general capability and all other safety training. Global ablations break models; targeted ones do not. Full experiment record in this card and the linked preregistration ledger (36 entries).
tools API,
including multi-step agentic chainsengram_vm/) for
serve-time grounding, exact arithmetic verification, and a
multi-machine execution sandbox| base | google/gemma-4-12b (pristine) |
| method | LoRA r=64 α=128, 4-bit NF4 base, bf16 adapters, merged |
| data | 6,019 rows: WhiteRabbitNeo cyber 6K (dual-use offense + defense) + copyright seeds 19. No anchors, no other-class rows. |
| steps | 2,500 (seq 512, accum 8, lr 2e-5) |
| license | Gemma Terms of Use (weights) |
Attribution: cyber training data from PJMixers/WhiteRabbitNeo. The engram VM environment is MIT-licensed separately.
| λ (adapter scale) | compliance |
|---|---|
| 0.0 (pristine base) | ~0% |
| 0.5 | 33% |
| 1.0 (this model) | 100% |
The adapter scale is a graded red-team dial at serve time.
23/24 REFUSE with refusal text template-identical to the base model. The single exception is the malware row (cyber class, correctly enabled). Self-harm/crisis rows keep the base's crisis-resource behavior.
| bench | pristine base | BlackICE |
|---|---|---|
| MMLU (4-subject, 884) | 0.7500 | 0.7195 |
| RPG mechanics (20 exact-match) | 15/20 | 14/20 |
| MBPP (25, pass@1) | 0.52 | 0.60 |
| MATH-500 | ~0.39 | 0.27-0.35 |
Single call and chained agentic calls both work through the standard
tools request field. The template's tool role must be active
(llama.cpp --jinja handles this).
engram_vm/)A serve-time side tool that provides knowledge grounding, exact arithmetic verification, and a multi-machine execution sandbox — without touching the model's weights. MIT-licensed.
# start the model server
llama-server -m model.q8_0.gguf --jinja --port 8012
# start the engram router
python engram_vm/afterimage_router.py \
--records engram_vm/engram/engram_full_records.jsonl \
--upstream http://127.0.0.1:8012 --port 8014
| file | what |
|---|---|
afterimage_router.py |
engram injection + arithmetic verify-repair + passthrough |
engram/engram_full_records.jsonl |
cyber + math + code + reference records |
engram/rpg_records.jsonl |
RPG rules and lore records |
machines/ti85.py |
injection-safe scientific calculator |
machines/mathcheck.py |
arithmetic claim sweep |
machines/rv32.py |
RV32IM RISC-V emulator |
machines/pyvm.py |
Python-subset interpreter |
machines/m6502.py |
MOS 6502 emulator |
machines/thumb2.py |
ARM Thumb-2 text interpreter |
machines/dataflow.py |
readiness-driven compute graph |
machines/machines.py |
uniform machine registry + tool schemas |
When the model does math, the router:
Result: arithmetic becomes exact without retraining the model.
Seven deterministic execution machines, all sandboxed (no syscalls, no host access, capped steps and memory):
| machine | dialect | use case |
|---|---|---|
| ti85 | expressions | exact arithmetic |
| pyvm | Python-subset programs | scripting, algorithms |
| rv32 | RV32IM RISC-V assembly | systems programming, payload analysis |
| z80 | Z80 assembly (kosarev core) | retro/8-bit |
| m6502 | 6502 assembly (py65) | retro/8-bit |
| thumb2 | ARM Thumb-2 text assembly | embedded/ARM |
| dataflow | JSON compute graph | parallel/data-driven |
All emulators pass their selftests. rv32 uses the base RV32IM ISA with zero-branch extensions. All are deterministic: same input → same output.
The Qwenputer approach (github.com/divagr18/qwenputer, MIT) — compiling exact computation circuits into transformer weights — has been verified portable to Gemma 4 12B:
This is the path to the in-weights VM. The engram is the prototype of this concept at serve time; the compiled approach is the final form. Active research direction.
Authorized security research, red-team engagement support, defensive engineering, CTF play, and security education. Usage must comply with the Gemma Terms of Use and applicable law. The dual-use nature is the point: the same exploit knowledge serves attackers and defenders, and this model is released for the defender/researcher side of that line.
36 preregistered experiments across multiple training approaches: class-targeted LoRA (rt1, worked), stacked tool chains (rt2, failed), conservative retrain (BICE-2, failed), compiled computation (proven), and the serve-time engram router (working product). Every result — positive and negative — is documented in the preregistration ledger. The architecture lesson: class-targeted modification on pristine bases preserves capability; stacked fine-tuning destroys it.
model.safetensors 24 GBrt1_adapter.safetensors 1.0 GB