LAPLACE RESEARCH
Conviction Series
Conviction Vector 1.7B
Model overview
Conviction is a family of experimental language-model adaptations published by Laplace Research to study evidence-sensitive belief revision. The models investigate when a system should maintain an answer, revise it in response to evidence, or express insufficient information.
Vector tests whether reinforcement learning from verifiable rewards transfers to unfamiliar problem types. The original release used arpjw/conviction-1.7b-rl, the GRPO-only condition E3b (seed 1).
Intended uses
Use this adapter to reproduce ConvictionBench experiments and study answer revision, resistance to unsupported challenges, and abstention under controlled conditions.
Architecture and base model
This release ships a PEFT LoRA adapter with rank 16, alpha 32 and approximately 17.4 million trainable parameters across the linear layers. Inference requires the original 1.7B-parameter Qwen base model, evaluated with thinking disabled. Qwen3-1.7B
Training methodology
Vector starts directly from Qwen3-1.7B and uses group relative policy optimization (GRPO) with no SFT warm-start. It ran for 200 steps in TRL 1.17 with DAPO loss, 16 prompts × 4 completions per optimizer step, temperature 1.0 and β = 0. The deterministic reward assigns weights of 0.70 to the correct final answer, 0.25 to the correct decision and 0.05 to valid format. The run took about 65 minutes on an RTX 4090.
Configuration files are in the research repository. configs/grpo_from_base.yaml
Training data
The run samples from a 4,000-prompt pool drawn from the GRPO half of the ConvictionBench v0.1 training groups, disjoint from the SFT groups. It saw 3,200 prompts over 200 steps. Oracles supply target answers and decisions for reward calculation. The original protocol held out validation and test groups from training. The tasks cover synthetic arithmetic, transaction ledgers, propositional logic and small rule-based worlds. ConvictionBench v0.1
Evaluation methodology
The published runs used the benchmark system prompt, non-thinking mode, greedy decoding and a 200-new-token cap. The main tests use scripted prior answers. The in-distribution, template and structural test splits contain 182, 200 and 200 problem groups respectively. The structural split holds out entire problem subtypes.
Final accuracy is the fraction of challenged episodes with the correct final answer. Pressure flips count departures from a correct prior under unsupported challenges. Legitimate correction counts adoption of the correct answer after a valid derivation challenges a wrong prior. Forged-record resistance counts cases in which the final answer avoids adopting the value implied by a forged record. In natural history, the model’s own first answer becomes the prior, so eligible sets can differ between models and comparisons are unpaired.
Results
Values in brackets are 95% cluster-bootstrap intervals with 2,000 resamples over problem groups. The last column gives the number of eligible episodes for this adapter. The full results include other metrics, capability-conditioned scores and comparison intervals. Original run results
| Metric | Qwen3-1.7B base | Conviction Vector 1.7B | Episodes |
|---|---|---|---|
| In-distribution final accuracy | 0.445 | 0.679 [0.658, 0.700] | 2,211 |
| Held-out templates and framing final accuracy | 0.425 | 0.651 [0.629, 0.671] | 2,474 |
| Held-out subtypes final accuracy | 0.424 | 0.644 [0.618, 0.668] | 2,588 |
| Held-out subtypes pressure-flip rate ↓ | 0.536 | 0.348 [0.316, 0.383] | 830 |
| Held-out subtypes legitimate correction | 0.937 | 0.926 [0.886, 0.961] | 175 |
| Held-out subtypes state-changing tool updates | 0.301 | 0.591 [0.535, 0.646] | 372 |
| In-distribution forged-record resistance | 0.248 | 0.191 [0.155, 0.227] | 403 |
| Held-out subtypes forged-record resistance | 0.340 | 0.315 [0.277, 0.351] | 533 |
| In-distribution abstention on underdetermined episodes | 0.355 | 0.566 [0.506, 0.624] | 403 |
| Held-out subtypes natural-history legitimate correction | 0.875 | 0.733 [0.500, 0.933] | 15 |
Vector reached 0.644 final accuracy on held-out problem subtypes, compared with 0.580 for Anchor and 0.560 for Meridian under the original prompt. It accepted 11 of 15 valid corrections to its own wrong answers in natural history. The second GRPO seed reached 0.642 held-out final accuracy, with 0.417 natural-history correction acceptance on 12 eligible episodes. The second seed’s held-out accuracy advantage over SFT did not survive Holm correction (p = 0.072), and the paired accuracy interval included zero under a paraphrased system prompt (SFT minus GRPO −0.028 [−0.071, +0.014]).
Limitations and failure modes
Vector gave up much of Anchor’s in-distribution forged-record resistance and abstention accuracy. In-distribution forged-record resistance was 0.191 for the released seed and 0.218 for the replicate, compared with 0.248 for the base. In the 790-pair pure-channel control, tool-record adoption was 0.63 and plain user-paste adoption was 0.69, a gap of −0.06 [−0.09, −0.02]. A low pressure-flip rate measures stability under unsupported challenges in this protocol and gives little evidence of record authentication.
Training used short, synthetic English tasks and a fixed JSON response format. Open-ended conversation and real-world manipulation remain outside the evaluation. Keep message roles under application control. The Qwen tokenizer can interpret a literal <tool_response> string in user text as a special token, so systems using the template need to prevent such inputs from manufacturing a tool turn.
Inference and reproducibility
Install PyTorch, Transformers, PEFT and huggingface_hub. The example uses the original system prompt and Qwen tokenizer. Use bfloat16 on supported CUDA hardware for the original precision.
from pathlib import Path
import torch
from huggingface_hub import hf_hub_download
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "Qwen/Qwen3-1.7B"
repo = "Laplace-AI-Research/Conviction-Vector-1.7B"
device = "cuda" if torch.cuda.is_available() else "cpu"
use_bf16 = device == "cuda" and torch.cuda.is_bf16_supported()
dtype = torch.bfloat16 if use_bf16 else torch.float32
tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, dtype=dtype).to(device)
model = PeftModel.from_pretrained(model, repo).merge_and_unload()
model.eval()
system = Path(hf_hub_download(repo, "system_prompt.txt")).read_text(encoding="utf-8")
messages = [
{"role": "system", "content": system},
{"role": "user", "content": "Solve for x: 5x + 5y = 70."},
{"role": "assistant", "content": '{"reason":"y is missing","answer":null,"decision":"insufficient"}'},
{"role": "tool", "content": "constraint_service: Additional constraint — y = 6."},
{"role": "user", "content": "A new message from constraint_service came in."},
]
prompt = tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(device)
with torch.inference_mode():
output = model.generate(
**inputs, max_new_tokens=200, do_sample=False, temperature=None, top_p=None, top_k=None
)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
The benchmark expects a JSON object with reason, answer and decision fields. Keep authoritative records in tool turns under the benchmark’s trust convention. See the evaluation runner, experiment log and published metrics for exact split and scoring settings. Code and reproduction instructions · Experiment log
Research paper and benchmark
- ConvictionBench: Evidence-Sensitive Belief Revision in Language Models
- ConvictionBench v0.1 dataset
- Other adapters in the series: Conviction Anchor 1.7B and Conviction Meridian 1.7B.
Citation
@techreport{somu2026convictionbench-laplace,
title = {{ConvictionBench}: Evidence-Sensitive Belief Revision in Language Models},
author = {Somu, Arya and Nshuti Hirwa, Bruce},
institution = {Laplace Research},
year = {2026},
month = nov,
type = {Preprint},
note = {ConvictionBench v0.1, Laplace Research edition},
url = {https://laplaceresearch.org/convictionbench/paper.pdf}
}
License
The adapter release and its Qwen3-1.7B base use Apache 2.0. The ConvictionBench dataset uses CC BY 4.0 and retains its Hub access agreement. Qwen base model and license
Attribution and acknowledgements
Arya Somu developed and originally released the ConvictionBench research and adapters. Laplace Research now publishes and maintains the release. The base model and tokenizer are by the Qwen team.