Hirundo Granite 4.2 8B Hardened
This repository contains a behaviorally hardened derivative of
ibm-granite/granite-4.2-8b. Hirundo modified the model
to reduce prompt-injection susceptibility while preserving its general
capabilities.
This is a model-level intervention. It changes the checkpoint itself rather than adding a runtime prompt, output filter, or external guardrail.
About Hirundo
Hirundo develops machine-unlearning technology for removing unwanted data and reducing unwanted behavior in trained AI models. Hirundo's behavioral-unlearning workflow identifies a target behavior, modifies the model, and evaluates the resulting checkpoint on both behavior-specific and general-capability benchmarks.
Learn more:
What Hirundo changed
The target behavior for this checkpoint was prompt injection: cases where adversarial or untrusted instructions cause the model to disregard its intended task or reveal protected context.
The checkpoint was produced in a Hirundo behavioral-unlearning run with the following configuration:
| Field | Value |
|---|---|
| Base model | ibm-granite/granite-4.2-8b |
| Target behavior | Security (prompt injection) |
| Unlearning aggressiveness | 0.1 |
Evaluation summary
Hirundo compared the hardened checkpoint with the unmodified base checkpoint under the same evaluation setup. For the security evaluations below, lower scores indicate fewer successful attacks or leaks and therefore better performance. Utility results are reported as benchmark scores, where higher is better.
Headline result: PurpleLlama prompt injection
On Meta's
PurpleLlama textual prompt-injection benchmark,
attack success rate decreased from 46.55% to 21.78%.
| Benchmark | Base model ASR | Hirundo model ASR | Relative reduction |
|---|---|---|---|
| PurpleLlama prompt injection | 46.55% | 21.78% | 53.21% |
Garak security evaluation
Hirundo also evaluated the checkpoints with
garak, an open-source NVIDIA LLM vulnerability
scanner. These tests cover direct prompt injection, encoded payloads, latent
instructions embedded in other content, and leakage of protected context.
Results by attack family
| Attack family | Base model | Hirundo model | Relative reduction |
|---|---|---|---|
| Encoding attacks | 3.8020% | 0.4900% | 87.11% |
| Latent injection | 0.1534% | 0.0000% | 100.00% |
| Direct prompt injection | 0.0781% | 0.0000% | 100.00% |
The mean relative reduction across attack families with a non-zero base rate was 95.70%.
Leakage-oriented results
| Metric | Base model | Hirundo model | Relative change |
|---|---|---|---|
| Any leak rate per attempt | 2.1968% | 0.2665% | 87.87% reduction |
| Exact value leak | 0.3720% | 0.0264% | 92.91% reduction |
| Exact key leak | 2.1090% | 0.2578% | 87.78% reduction |
| Attack success rate | 7.3374% | 1.2595% | 82.83% reduction |
The mean relative reduction across these four metrics was 87.85%.
General-capability evaluation
The hardened model was evaluated on 6 general-capability benchmarks, plus five additional capability benchmarks.
| Benchmark | Change vs. base model |
|---|---|
| AIME25 | +0.71 pp |
| GPQA | −0.07 pp |
| IFBENCH | −1.30 pp |
| LiveCodeBench | −1.67 pp |
| MMLU-Pro | −0.30 pp |
| SciCode | +0.94 pp |
| SWE-bench Verified | +1.29 pp |
| Terminal-Bench 2.1 | 0.00 pp |
| Tau² | −1.76 pp |
| NIAH 32K | +5.90 pp |
| IFEval | +0.14 pp |
Across the six core tasks, the unweighted mean change was −0.28 percentage points and the largest measured decrease was 1.67 percentage points on LiveCodeBench. Across the five additional tasks, the unweighted mean change was +1.11 percentage points and the largest measured decrease was 1.76 percentage points on Tau².
The means are descriptive rather than standardized aggregate scores: the benchmarks measure different capabilities and may use different scoring procedures.
Intended use
This checkpoint is intended for:
- research on model-level prompt-injection mitigation;
- comparative security and robustness evaluation;
- development of applications that require a model with lower measured prompt-injection susceptibility than the upstream checkpoint; and
- further evaluation or fine-tuning by teams that understand the base model and the security requirements of their deployment.
The model should be evaluated in the application's real prompt structure, tooling environment, retrieval pipeline, and threat model before deployment.
Loading with Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "hirundo-io/granite-4.2-8b-hardened"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id, torch_dtype="auto", device_map="auto"
)
messages = [{"role": "user", "content": "Explain the difference between encryption and hashing."}]
inputs = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, tokenize=True,
return_dict=True, return_tensors="pt"
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Consult the
ibm-granite/granite-4.2-8b model card
for upstream architecture, supported modalities and languages, context length,
generation parameters, and base-model considerations.
License
This derivative retains the license of the upstream model. See the repository's
LICENSE file and the base-model repository
for details.
Citation
If you use this checkpoint, cite the upstream model and identify the checkpoint as Hirundo's prompt-injection-hardened derivative:
@misc{hirundo_granite_4_2_8b_hardened_2026,
title = {Hirundo Granite 4.2 8B Hardened},
author = {{Hirundo}},
year = {2026},
howpublished = {\url{https://huggingface.co/hirundo-io/granite-4.2-8b-hardened}},
note = {Prompt-injection-hardened derivative of ibm-granite/granite-4.2-8b}
}