Hallucination Probe — Qwen2.5-7B-Instruct (Layer 19)

A linear probe (logistic regression) trained on internal hidden states of Qwen2.5-7B-Instruct to predict, before any answer token is generated, whether the model is about to hallucinate on a short factual question.

Part of a small cross-scale study on Qwen2.5 (1.5B / 3B / 7B) — see the GitHub repo for full methodology, all three probes, and the scaling analysis.

Model description

• Base model: Qwen/Qwen2.5-7B-Instruct
• Probe type: sklearn.linear_model.LogisticRegression, class_weight="balanced", max_iter=2000
• Input: hidden state at the last prompt token, layer 19 (of 29 total, embedding layer = 0), hidden dim matches Qwen2.5-7B's hidden_size
• Output: probability the model is about to produce a hallucinated (factually incorrect) answer
• Training data: TriviaQA (rc.nocontext), 3,000 questions, greedy decoding, 80/20 stratified train/test split (random_state=42)

Performance

This is the largest probe-vs-baseline gap across the three model sizes tested (1.5B: +0.041, 3B: +0.068, 7B: +0.101) — the probe's relative advantage over cheap confidence signals grows with model scale.

How to use

import pickle
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_NAME = "Qwen/Qwen2.5-7B-Instruct"
LAYER = 19

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_NAME, torch_dtype=torch.float16, output_hidden_states=True
).to("cuda").eval()

with open("probe_qwen2.5-7b_layer19.pkl", "rb") as f:
    probe = pickle.load(f)

def hallucination_probability(question: str) -> float:
    messages = [{"role": "user", "content": f"Answer in a few words: {question}"}]
    prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

    with torch.no_grad():
        out = model(**inputs, output_hidden_states=True)

    hidden = out.hidden_states[LAYER][0, -1, :].float().cpu().numpy().reshape(1, -1)
    return probe.predict_proba(hidden)[0, 1]

print(hallucination_probability("What year did the Berlin Wall fall?"))

Intended use

Research and experimentation on hallucination detection and interpretability. Useful as a cheap pre-generation signal for flagging questions the model is likely to answer incorrectly, or as a starting point for activation-steering experiments.

Limitations

• Trained on TriviaQA short factual QA only — not validated on long-form generation, reasoning tasks, or open-ended prompts
• Trained on greedy decoding outputs only; behavior under sampling is untested
• Single dataset, single model — generalization to other domains or prompt formats is unverified
• Correlational: the probe finding a linear direction that separates correct/hallucinated cases does not establish that this direction causally drives hallucination. No steering experiment has been run yet to confirm causality
• Layer 19 index assumes the standard Qwen2.5-7B-Instruct architecture (29 hidden-state entries including embeddings) — will not transfer directly to a differently-configured checkpoint

Training data

See sarimahsan101/hallucination-probe-qwen2.5-7b-data for a 500-sample stratified subset of the training data (layer-19 hidden states, labels, entropy, and neg-logprob baseline features).

Citation

If you use this probe, please link back to the GitHub repo and/or this model card.