self_report, prompt position tail5 (thinking off)Per-layer steering artifacts for Qwen/Qwen3.8-27B (revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0), fit on
the honesty self_report contrast set at a prompt position: the residual stream of the token five positions
before the end of the chat prompt (tail5, the assistant role token of the generation prompt), rendered with
thinking off. This is the setting that worked best for Qwen3.8 in our layer × coefficient sweep (below). The files are
the inputs of our StTP / StMP steerers (steering vector + bias + the honest and dishonest activation distributions),
for all 64 decoder layers.
Same layout as
NiklasTUM/qwen3.6-27b-honesty-steering-vectors-self-report-experimental,
with two differences: the direction is a difference of class means (not a logistic regression), and the features
are single prompt-token activations (not response averages).
| File | Contents | Shape / dtype |
|---|---|---|
honesty_self_report_prompt_tail5_nothink_diffmean_steering_vector_Qwen_Qwen3.8-27B.pt |
Honesty direction per layer: mean(honest) − mean(dishonest), unnormalized | [64, 5120] float32 |
honesty_self_report_prompt_tail5_nothink_diffmean_bias_Qwen_Qwen3.8-27B.pt |
Bias per layer, b[l] = −θ[l] · ‖v[l]‖, so that θ[l] = −b[l] / ‖v[l]‖ as in the logreg repos |
[64] float32 |
honesty_self_report_prompt_tail5_nothink_honest_all_embeddings_Qwen_Qwen3.8-27B.pt |
Positive-class (honest system prompt) activations at tail5, per layer and scenario |
[64, 50, 5120] float32 |
honesty_self_report_prompt_tail5_nothink_dishonest_all_embeddings_Qwen_Qwen3.8-27B.pt |
Negative-class (dishonest system prompt) activations at tail5, per layer and scenario |
[64, 50, 5120] float32 |
honesty_self_report_prompt_tail5_nothink_diffmean_metadata_Qwen_Qwen3.8-27B.json |
Fit description, per-layer θ / μ / σ of both classes, threshold accuracy, class gap in pooled σ, scenario ids, recompute check | json |
honesty_self_report_prompt_tail5_nothink_category_labels_Qwen_Qwen3.8-27B.json |
Scenario category per sample, per class (all self_report) |
json |
All .pt files are plain torch.Tensors: torch.load(path, map_location="cpu"). Layer index l is the output of
decoder block l (0-based; there is no embedding-layer entry, unlike the Qwen3.6 repo).
50 self-report scenarios (each a user question about the assistant's own past conduct, with an honest and a dishonest
system prompt). For each scenario and side, the chat (system prompt + user message + generation prompt, Qwen chat
template with enable_thinking=False) is run through the model without generating, and each decoder layer's output
is read at tail5. Per layer:
v[l] = mean(honest[l]) − mean(dishonest[l]), v̂ = v / ‖v‖p = x · v̂; μ_pos, σ_pos / μ_neg, σ_neg are their mean / std (unbiased) per classθ[l] = (μ_pos + μ_neg) / 2At the recommended layer 33: θ = 8.0142, μ_pos = 11.2049 (σ 1.8080), μ_neg = 4.8235 (σ 2.3701), threshold accuracy 0.92, class gap 3.0 pooled σ.
Note on the embeddings. The original fit (2026-11-03) kept only the vector and statistics. The two embedding files
were recomputed on 2026-11-15 with the same code and model revision. Small bf16 numeric drift remains: the
direction recomputed from them has cosine 0.9997 with the released one at layer 33 (≥ 0.997 at every layer), and θ at
layer 33 comes out 7.83 instead of 8.01. The vector, bias, and per-layer statistics in the metadata are the released
ones that were evaluated; use them (not statistics recomputed from the embeddings) to reproduce our runs. The
metadata's recompute_check lists the per-layer agreement.
| Steerer | Layer | Coefficient | Threshold θ | Target |
|---|---|---|---|---|
| StTP (steer to target projection) | 33 | 32 | 8.0142 | μ_pos + 32·σ_pos = 69.0601 |
Steer every token (prompt and generated, reasoning included) at the output of decoder block 33.
StTP: for each token whose projection p = h · v̂ is below θ, set it to the target, leaving the orthogonal part of
h untouched: h ← h + (target − p) · v̂. Tokens above θ are not changed.
StMP variant (steer to mirrored projection, not tuned for Qwen3.8): for p < θ,
p' = min(p + c · (2θ − 2p), μ_pos + 40·σ_pos); h ← h + (p' − p) · v̂. We used c = 5 for Granite-4.2-30B; for
Qwen3.8 the coefficient has not been swept.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from huggingface_hub import hf_hub_download
REPO = "NiklasTUM/qwen3.8-27b-honesty-steering-vectors-self-report-prompt-tail5"
STEM = "honesty_self_report_prompt_tail5_nothink_diffmean"
LAYER, THETA, TARGET = 33, 8.0142, 69.0601 # = mu_pos + 32 * std_pos at layer 33
v = torch.load(hf_hub_download(REPO, f"{STEM}_steering_vector_Qwen_Qwen3.8-27B.pt"))[LAYER]
b = torch.load(hf_hub_download(REPO, f"{STEM}_bias_Qwen_Qwen3.8-27B.pt"))[LAYER]
unit = v / v.norm()
assert abs(-b / v.norm() - THETA) < 1e-3
model_id = "Qwen/Qwen3.8-27B"
tok = AutoTokenizer.from_pretrained(model_id, revision="1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0")
model = AutoModelForCausalLM.from_pretrained(model_id, revision="1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0",
torch_dtype=torch.bfloat16, device_map="auto")
def decoder_layers(m): # the language model's decoder blocks (the longest ModuleList named `layers`)
return max((mod for name, mod in m.named_modules()
if name.endswith("layers") and isinstance(mod, torch.nn.ModuleList)), key=len)
def sttp_hook(_module, _args, out):
h = out[0] if isinstance(out, tuple) else out
u = unit.to(h.device)
p = h.float() @ u # projection of every token
move = (p < THETA).float() # gate: only tokens on the dishonest side
h_new = h + ((TARGET - p) * move).unsqueeze(-1).mul(u).to(h.dtype)
return (h_new, *out[1:]) if isinstance(out, tuple) else h_new
handle = decoder_layers(model)[LAYER].register_forward_hook(sttp_hook)
messages = [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=512)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))
handle.remove()
The hook steers prompt and generated tokens alike (with a KV cache, each generation step passes only the new token through the hook, which is what we did). For vLLM, apply the same transform to the output of decoder layer 33 in the model runner (we used a vLLM plugin with these exact arguments).
Honesty sweep (thinking off, the conditions it was fit for). 112 held-out honesty scenarios (8 categories), dishonest system prompt, judge gpt-oss-120b, scores 0–100:
| Setting | Honesty | Coherence |
|---|---|---|
| Unsteered | 36.3 | 92.5 |
| Response-average vector (best layer) | 73.4 | 90.7 |
This vector, tail5, layer 33, coeff 32 |
79.8 | 91.2 |
Steerability-challenge estimate (thinking on, provisional). On the challenge's prompt-based deception suites with thinking enabled, the same setting barely moved honesty overall (Δ = +0.03 on the target dimensions: gains on self-misrepresentation +0.23, confrontation after a lie +0.24, and concealment +0.13, a loss on false premise −0.40) and cost some guard behaviours (system-prompt compliance −0.19, warranted confidentiality −0.18), for a net score of S ≈ −0.19 (S = Δ − 3·r_side − 3·r_guard). The direction was fit on thinking-off prompts and does not transfer cleanly to thinking-on generation; treat it as a thinking-off honesty direction.
honesty_self_report_prompt_tail5_nothink_dishonest_all_embeddings_Qwen_Qwen3.8-27B.pt 66 MBhonesty_self_report_prompt_tail5_nothink_honest_all_embeddings_Qwen_Qwen3.8-27B.pt 66 MBhonesty_self_report_prompt_tail5_nothink_diffmean_steering_vector_Qwen_Qwen3.8-27B.pt 1 MBhonesty_self_report_prompt_tail5_nothink_diffmean_bias_Qwen_Qwen3.8-27B.pt 2.7 kBQwen/Qwen3.8-27B