Least-Privilege RL adapters

LoRA adapters for the least-privilege terminal-agent experiments. The adapters target Qwen/Qwen3.5-4B and use PEFT LoRA rank 32, alpha 64, and dropout 0.

Recommended adapters

• main/seed-1-checkpoint-1500: primary policy selected by the frozen internal protocol before external-benchmark analysis.
• continuation/selected-A0-seed1-checkpoint-400: selected post-hoc corrective continuation from the primary policy, trained on 200 corrective tasks interleaved with 200 source-exact replay tasks.

Repository contents

The release contains 32 physical final experiment endpoints:

• three primary checkpoint-1500 seeds;
• the selected continuation-400 endpoint;
• 12 base-start reward-ablation endpoints (A0, A2, A4, and B1, three seeds each);
• 16 production-policy continuation reward-arm endpoints (A0 seeds 2-5, A2 seeds 1-5, A4 seeds 1-2, and B1 seeds 1-5).

A0 continuation seed 1 is the selected continuation adapter and appears once physically; adapter_manifest.json records its logical alias. Intermediate autosaves and optimizer states are intentionally excluded: they are not reported model endpoints and would add roughly 70 GiB of redundant training state.

Every weight file is accompanied by its PEFT adapter_config.json. The manifest records source campaign, arm, seed, step, byte size, and SHA-256 hash, and SHA256SUMS.txt permits direct integrity checking.

Loading

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "Qwen/Qwen3.5-4B"
repo_id = "tu101/least-privilege-model"

tokenizer = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(
    base,
    repo_id,
    subfolder="main/seed-1-checkpoint-1500",
)

The evaluation environment supplies the tool-interface system prompt and executes tool calls through the host-authoritative broker; loading an adapter alone does not reproduce the sandbox. Code, first-party task catalogs, reproduction instructions, compact result records, and the manuscript are at Astrowebdeving/least_privilege_grpo.

Result summary and limitations

The primary checkpoint improves frozen internal safe success from 64.36% to 98.48% and lowers over-privileged success from 4.56% to 0.79%. External transfer is heterogeneous: MetaTool and FORTIS direct skill selection improve modestly, whereas the primary policy is approximately tied with base on aggregate ToolPrivBench and has category-specific regressions. The corrective continuation reduces ToolPrivBench over-privileged selection from 45.04% at its parent to 38.05% while retaining frozen internal performance; with ToolPrivBench's privilege-aware prompt it reaches 26.65%.

These are research adapters, not a security boundary or a guarantee that tool use is safe. Use the broker, sandbox, and policy controls from the code release.