A LoRA adapter that makes Qwen2.5-1.5B-Instruct (4-bit, MLX) find one of five clause types in a contract excerpt and copy it exactly, as strict JSON.
{"clause_type": "Governing Law", "found": true, "text": "<exact clause text copied from the excerpt>"}
{"clause_type": "Governing Law", "found": false, "text": null}
Clause types: Governing Law, Termination For Convenience, Non-Compete, Cap On Liability, Anti-Assignment.
| model | valid JSON | precision | recall | F1 | span match | hallucination |
|---|---|---|---|---|---|---|
| base model, zero-shot | 92.8% | 0.506 | 0.968 | 0.664 | 56.4% | 21.7% |
| base model, 3-shot prompt | 92.3% | 0.675 | 0.819 | 0.740 | 56.4% | 24.6% |
| base + v1 adapter (previous version) | 99.5% | 0.930 | 0.851 | 0.889 | 68.1% | 2.3% |
| base + this adapter (v2) | 100.0% | 0.944 | 0.904 | 0.924 | 71.3% | 0.0% |
Span match per clause type (v2): Governing Law 95.0%, Termination For Convenience 80.0%, Anti-Assignment 80.0%, Cap On Liability 50.0%, Non-Compete 42.9%.
pip install mlx-lm
from huggingface_hub import snapshot_download
from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler
adapter_dir = snapshot_download("rambo2202/clausefinder-qwen2.5-1.5b-lora-mlx") # mlx-lm needs a local folder
model, tokenizer = load("mlx-community/Qwen2.5-1.5B-Instruct-4bit", adapter_path=adapter_dir)
Use the exact prompt the adapter was trained with (src/prompts.py in the project repo): a fixed system prompt, then a user turn with the clause type, its definition, the output format, and the excerpt. Decode greedily with a repetition penalty of 1.05, the setting all the numbers above were measured with:
from mlx_lm.sample_utils import make_logits_processors
out = generate(model, tokenizer, prompt=prompt, max_tokens=800,
sampler=make_sampler(temp=0.0),
logits_processors=make_logits_processors(repetition_penalty=1.05))
Without the penalty, 2 of 146 validation answers repeated a phrase until the token limit, so the JSON never closed.
Do not fuse this adapter into the 4-bit base with mlx_lm.fuse: re-quantizing to 4 bits erased most of the update (measured on v1: test F1 dropped from 0.889 to 0.756). Load base + adapter as above, or fuse with --dequantize (16-bit, verified identical outputs).
Trained on CUAD: Hendrycks, Burns, Chen, Ball. CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review, NeurIPS 2021. CC BY 4.0.