Qwen2.5-1.5B CUAD Clause Extractor

A 1.5B parameter model that reads a commercial contract and returns a JSON object with the passages belonging to each of the 41 clause categories of CUAD: parties, dates, renewal terms, governing law, termination rights, restrictive covenants, licences, liability caps, insurance, audit rights and more. Every value is a verbatim span of the contract.

It is Qwen2.5-1.5B-Instruct fine-tuned with QLoRA on the CUAD training contracts, with the LoRA weights merged into the 16-bit base.

• Code, training and evaluation: github.com/Z-Nadjib/ClauseLM-ContractExtraction-CUAD
• Demo: huggingface.co/spaces/Z-Nadjib/ClauseLM-demo

Results

102 contracts of the official CUAD test split, document level. A span is correct when its word-level Jaccard similarity with a gold span of the same field is at least 0.5.

Latency is measured on one RTX 5080 (16 GB) with transformers in bf16, all windows of a contract in one batch. At an on-demand rental price of USD 0.40 per GPU hour this is USD 0.0020 per contract; the electricity cost at 360 W and USD 0.25 per kWh is USD 0.00045.

Best fields: governing law (86 F1), document name (87), expiration date (83), renewal term (83). Weakest fields: post-termination services (18), non-disparagement (8), third party beneficiary (9), most favored nation (0, 3 test spans).

Output format

Given a contract excerpt of up to about 6,000 characters, the model returns a compact JSON object containing only the fields it found. Values are lists of strings.

{"document_name": ["SUPPLY CONTRACT"], "insurance": ["To be covered by the Seller for 110% invoice value against All Risks and War Risk."]}

Field keys: document_name, parties, agreement_date, effective_date, expiration_date, renewal_term, notice_period_to_terminate_renewal, governing_law, most_favored_nation, non_compete, exclusivity, no_solicit_of_customers, competitive_restriction_exception, no_solicit_of_employees, non_disparagement, termination_for_convenience, rofr_rofo_rofn, change_of_control, anti_assignment, revenue_profit_sharing, price_restrictions, minimum_commitment, volume_restriction, ip_ownership_assignment, joint_ip_ownership, license_grant, non_transferable_license, affiliate_license_licensor, affiliate_license_licensee, unlimited_all_you_can_eat_license, irrevocable_or_perpetual_license, source_code_escrow, post_termination_services, audit_rights, uncapped_liability, cap_on_liability, liquidated_damages, warranty_duration, insurance, covenant_not_to_sue, third_party_beneficiary.

How to use

The model was trained with a fixed system prompt that lists the field definitions. The simplest way to get identical inputs, long-document chunking, grammar-constrained decoding and merging is the project package:

from contract_extractor.inference.base import ChunkedExtractor
from contract_extractor.inference.hf_backend import HFBackend

backend = HFBackend("Z-Nadjib/ClauseLM-Qwen2.5-1.5B", constrained=True)
extractor = ChunkedExtractor(backend, chunk_size=6000, chunk_overlap=600, batch_size=8)
result = extractor.extract(open("contract.txt").read())
print(result.extraction)

With transformers only, for one excerpt:

from transformers import AutoModelForCausalLM, AutoTokenizer
from contract_extractor.prompts import build_messages  # or copy SYSTEM_PROMPT from the repository

tokenizer = AutoTokenizer.from_pretrained("Z-Nadjib/ClauseLM-Qwen2.5-1.5B")
model = AutoModelForCausalLM.from_pretrained("Z-Nadjib/ClauseLM-Qwen2.5-1.5B", dtype="auto", device_map="auto")
prompt = tokenizer.apply_chat_template(build_messages(excerpt), tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
output = model.generate(**inputs, max_new_tokens=2048, do_sample=False, repetition_penalty=1.0)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Use greedy decoding without repetition penalty, as set in the bundled generation config: the outputs copy the input text, and a repetition penalty degrades copying.

Training

Evaluation

The full protocol, per-field scores and the constrained decoding ablation are in the project repository.

Limitations

• Trained on English contracts filed with the US SEC. Other languages, jurisdictions and document types are out of distribution.
• Spans are verbatim and not normalized (dates are returned as written).
• Clause categories with few training examples (for example Source Code Escrow or Unlimited/All-You-Can-Eat License) are less reliable.
• The output is an aid for contract review, not legal advice.

License

Apache 2.0, inherited from Qwen2.5-1.5B-Instruct. Training data: CUAD, CC BY 4.0, The Atticus Project.

Citation

@article{hendrycks2021cuad,
  title   = {CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review},
  author  = {Hendrycks, Dan and Burns, Collin and Chen, Anya and Ball, Spencer},
  journal = {NeurIPS Datasets and Benchmarks},
  year    = {2021}
}