A LoRA adapter to find bugs in code. It reasons in a few sentences, writes Verdict: break or Verdict: no break, and after a break names the defect and quotes the code.
Compared with the base Qwen3.8-27B on 100 functions from repositories it never saw in training:
| Base | Maris Piper | |
|---|---|---|
| Ranks buggy code above clean code (AUC, 0.5 is chance) | 0.63 | 0.81 |
| Real bugs flagged | 38 of 61 | 43 of 61 |
| Clean functions correctly passed | 23 of 39 | 27 of 39 |
| Flags that name the exact defect later fixed | 2 | 6 |
| Output needed to reach a verdict | up to 4,752 tokens | under 400 tokens |
Verdict: line. The base model often runs for thousands of tokens before deciding.Full results and caveats are under Results.
The adapter was trained on this one format. Other formats are untested.
System message:
You are reviewing one function from a pull request. The function is shown first, then the pull request's changes to its file as a unified diff. First reason in a few sentences: what the change did, what the function promises its callers and the rest of the system, and whether some concrete sequence of events breaks that promise. Then write a line reading exactly `Verdict: break` or `Verdict: no break`. After `Verdict: break`, state the break in one to three sentences and quote the code where it goes wrong.
User message: a heading with the file path, the whole function, then the diff of that file in a fenced block.
### src/path/to/file.ts
<the function as it is after the change>
### Changes to this file in the pull request
```diff
<unified diff of the file>
```
Thinking mode is off (enable_thinking=False). Answers run to about 95 words; 400 new tokens is enough.
The adapter was trained and evaluated on the base model loaded in 4-bit. Loading the base in 16-bit is untested.
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
BASE = "Qwen/Qwen3.8-27B"
ADAPTER = "cameronpyne-smith/maris-piper-bug-finder-qwen3.8-27b-lora"
SYSTEM = "You are reviewing one function from a pull request. The function is shown first, then the pull request's changes to its file as a unified diff. First reason in a few sentences: what the change did, what the function promises its callers and the rest of the system, and whether some concrete sequence of events breaks that promise. Then write a line reading exactly `Verdict: break` or `Verdict: no break`. After `Verdict: break`, state the break in one to three sentences and quote the code where it goes wrong."
quantisation = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True)
tokeniser = AutoTokenizer.from_pretrained(BASE)
base = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=quantisation, dtype=torch.bfloat16, device_map="cuda")
model = PeftModel.from_pretrained(base, ADAPTER).eval()
path = "src/example.ts"
function = "..."
diff = "..."
prompt = f"### {path}\n{function}\n\n### Changes to this file in the pull request\n```diff\n{diff}\n```"
messages = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": prompt}]
ids = tokeniser.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=False, enable_thinking=False).to("cuda")
with torch.no_grad():
output = model.generate(ids, max_new_tokens=400, do_sample=False, pad_token_id=tokeniser.pad_token_id)
print(tokeniser.decode(output[0, ids.shape[1]:], skip_special_tokens=True))
Needs a GPU with about 20 GB of memory.
Validation set is 100 functions, 61 bugged, 39 clean. The sample is 1 greedy answer and 4 sampled at temperature 0.7 and the score is the mean of the 5 runs.
| Measurement | Base Qwen3.8-27B | With this adapter |
|---|---|---|
| Ranking, AUC ↑ | 0.631 | 0.814 |
| Real bugs flagged (of 61) ↑ | 38 | 43 |
| …of those, naming the defect that was later fixed ↑ | 2 | 6 |
| Clean functions passed (of 39) ↑ | 23 | 27 |
| Balanced accuracy ↑ | 0.606 | 0.699 |
| Tokens needed to reach a verdict ↓ | up to 4,752 | under 400 |
↑ higher is better, ↓ lower is better, bold marks the better result.
How to read this:
1,177 examples from merged pull requests in 34 public, permissively licensed repositories, about 58% TypeScript and 42% C#. 759 functions containing a bug that was later fixed, and 418 with no later fix. Each is paired with a short written review ending in a verdict. Sources include ASP.NET Core, Entity Framework Core, Vue, Vite, NestJS and other widely used C# and TypeScript projects. The dataset is not published.
The 100 validation functions come from three other repositories (zod, Humanizer, Quartz.NET).
| Method | supervised fine-tuning on the answer tokens only |
| LoRA | rank 16, alpha 32, no dropout, on the attention and feed-forward projections |
| Base precision | 4-bit (nf4, double quantisation, bfloat16 compute) |
| Schedule | 1 pass over the data, 166 optimiser steps, peak learning rate 1e-4 with warm-up and cosine decay |
| Validation loss | 1.31 before, 0.87 after (nats per token; lower is better) |
The LoRA layers, optimiser and training loop are written from scratch in couch.
Qwen/Qwen3.8-27B