Veyra Decide 9B (Legal & General)

Veyra Decide 9B is a legal decision model with general capabilities. It answers typed questions about a document (multiple choice, yes/no, ordinal score) with calibrated probabilities.

Use

from transformers import AutoModel

model = AutoModel.from_pretrained("judicialmind/Veyra-Decide-9B-Legal-General", trust_remote_code=True)
result = model.system_one(
    state="Recipient shall not disclose Confidential Information to any third party without prior written consent.",
    questions={
        "third_party_sharing": {
            "type": "noul",
            "instructions": "Does the clause allow sharing confidential information with third parties?",
            "criteria": {"false": "No, sharing is not allowed.", "true": "Yes, sharing is allowed."},
        }
    },
)
print(result["answers"]["third_party_sharing"])

Requires transformers>=5.17, torch and safetensors. Maximum input: 32,768 tokens.

Results

Results were measured with inputs up to 16,384 tokens.

Legal (accuracy, %)

Benchmark	Questions	Veyra Decide 9B	Jev	Clef	Clef-flash	Perplexity
ContractNLI	2,091	89.7	77.4†	87.8	88.7	80.7
CUAD	3,895	84.3	89.8	91.1	87.8	92.2
CUAD, contracts within 16K tokens	3,444	95.4	90.5	91.8	88.7	92.9
LEDGAR	3,000	80.2	74.7	73.6	72.2	80.6
CaseHOLD	3,600	83.4	77.6	75.9	69.5	86.7
UNFAIR-ToS	12,856	98.6	97.0	97.4	97.9	98.7

† Published Jev-Benchmark answers.

General (accuracy, %)

Benchmark	Questions	Veyra Decide 9B	Jev	Clef	Clef-flash	Perplexity
BoolQ	3,270	91.2	92.0	91.3	91.0	92.8
MNLI	3,000	91.3	87.2	89.6	89.1	92.8
MNLI mismatched	2,996	91.8	86.8	89.3	89.9	92.8
ARC-Challenge	1,172	95.1	97.7	97.7	98.1	96.8
ARC-Easy	2,376	98.4	99.4	99.2	99.6	99.4

Long legal documents (accuracy, %)

Benchmark	Questions	Veyra Decide 9B	Jev	Clef	Clef-flash	Perplexity
LexGLUE ECtHR-A	10,000	92.4	92.7	94.5	93.0	93.4
LexGLUE ECtHR-B	10,000	94.0	91.9	94.3	93.4	94.3
LexGLUE SCOTUS	1,317	57.3	71.6	71.1	67.6	78.7
LexGLUE SCOTUS, opinions within 16K tokens	989	76.3	71.8	71.0	67.1	79.3
MAUD	6,651	64.9	64.4	64.0	60.9	66.7
JevBench	1,938	89.3	90.5	90.5	88.1	92.0

CUAD clause retrieval over contract paragraphs (mean AUPR, 40 clause types)

Veyra Decide 9B	Jev	Clef	Clef-flash	Perplexity
0.616	0.534	0.521	0.466	0.554

LegalBench (mean task accuracy, %)

Task group	Tasks	Veyra Decide 9B	Jev	Clef	Clef-flash	Perplexity
CUAD clauses	38	90.7	84.4	85.4	83.0	85.1
MAUD	34	72.2	79.3	79.2	72.1	76.4
ContractNLI	14	96.3	92.3	94.5	92.6	93.1
Learned Hands	16	88.1	82.5	88.2	86.8	88.6
Supply chain	10	82.4	77.4	76.9	80.9	77.6
OPP-115 privacy	9	85.4	83.4	83.9	80.8	83.4
Diversity	6	75.9	92.2	82.7	74.1	94.1
Other	27	78.3	76.9	75.8	70.7	79.5
All tasks	154	83.3	82.3	82.7	79.1	83.1

Robustness (accuracy, %)

Test	Veyra Decide 9B	Jev	Clef	Clef-flash	Perplexity
BoolQ, negated question	90.8	90.4	83.1	53.5	84.8
BoolQ, swapped yes/no descriptions	91.0	9.8	91.0	91.2	92.9
CUAD, negated question (descriptions rewritten to match)	84.2	90.9	88.7	87.1	90.0
CUAD, swapped yes/no descriptions	84.4	11.3	90.9	88.3	92.4
UNFAIR-ToS, negated question (descriptions rewritten to match)	99.1	97.0	97.7	97.2	98.4
UNFAIR-ToS, swapped yes/no descriptions	98.8	2.5	97.5	97.2	98.6
ContractNLI, random option ids	89.6	77.4	87.8	87.9	80.6
ContractNLI, shuffled options	89.6	78.2	87.8	88.7	81.0
CaseHOLD, random option ids	83.2	77.9	75.2	69.4	86.4
CaseHOLD, shuffled options	83.2	76.8	75.9	69.5	86.3
MNLI, random option ids	91.2	86.9	89.0	89.4	93.0
MNLI, shuffled options	91.3	87.3	89.6	89.1	92.9
ARC-Challenge, random option ids	94.9	97.4	97.8	98.0	96.8
ARC-Challenge, shuffled options	95.2	97.8	97.7	98.1	97.3

Decision Index (13 public benchmarks)

Benchmark	Metric	Veyra Decide 9B	Jev	Clef	Clef-flash	Perplexity
BANKING77	F1	84.3	79.7	94.1	90.8	80.6
CLINC150	F1	90.3	89.3	97.4	66.9	91.5
ContractNLI	F1	84.3	71.7	81.3	84.5	78.4
ANLI	F1	64.5	74.8	70.0	59.1	74.3
MMLU	accuracy	76.3	91.7	90.2	91.9	87.7
ARC-Easy	accuracy	98.2	99.3	99.0	99.5	99.3
ARC-Challenge	accuracy	95.1	97.8	97.8	98.2	97.0
WinoGrande	accuracy	87.6	92.0	93.5	97.5	92.4
HellaSwag	accuracy	91.9	94.5	98.2	98.6	93.2
NLI4CT	F1	80.5	84.1	83.0	78.6	84.9
MMLU-Pro	accuracy	53.6	82.7	66.0	65.2	71.0
BBH	accuracy	70.3	92.9	73.6	68.9	84.0
RAGTruth	F1	51.6	76.5	79.5	35.6	83.1
Mean of 13		79.1	86.7	86.4	79.6	86.0

Outside-system figures: public Decision Index leaderboard.

ContractNLI against Jev-Benchmark systems (2,091 judgments, accuracy, %)

System	Its accuracy	Veyra Decide 9B
Gemini Pro	83.2	89.7
Terra	82.6	89.7
Gemini Flash-Lite	82.5	89.7
Claude Sonnet 5	82.4	89.7
Astra	81.8	89.7
Claude Haiku 4.5	79.4	89.7
Qwen 9B	79.1	89.7
Luna	79.0	89.7
Jev	77.4	89.7
Qwen 4B	74.2	89.7

Outside-system figures: published Jev-Benchmark answers.

Jev harness tasks (Jev's scorer, primary metric x100)

Task	Metric	Veyra Decide 9B	Jev	Clef	Clef-flash	Perplexity
afrixnli	accuracy	56.5	62.8	53.4	49.2	70.8
ag_news	accuracy	88.4	87.5	89.3	89.3	89.8
agb_de	f1	10.3	20.4	17.9	7.0	23.5
anli	accuracy	64.6	74.3	62.1	57.1	72.5
arc	accuracy	96.8	98.5	98.6	99.2	98.0
art	accuracy	83.2	83.9	88.3	84.3	88.2
banking77	accuracy	84.0	79.2	94.0	95.2	82.0
belebele	accuracy	72.0	87.5	82.3	73.5	85.7
bigbench	accuracy	75.7*	83.7*	77.5*	72.2*	83.6
boolq	accuracy	90.7	91.0	91.2	91.0	92.3
ceval	accuracy	75.6	84.5	80.0	77.7	84.2
clinc150	accuracy	90.2	89.8	96.7	98.3	90.8
commonsense_qa	accuracy	86.1	88.2	89.4	92.9	88.6
emotion	accuracy	56.6	58.5	59.3	58.1	61.3
financial_phrasebank	accuracy	78.9	73.0	72.6	74.5	85.3
go_emotions	macro f1	23.4	24.1	46.5	50.3	35.3
hellaswag	accuracy	93.7	95.5	98.4	98.8	94.2
helpsteer2	mean spearman	40.5	41.2	43.5	38.6	50.8
imdb	accuracy	95.3	96.2	96.8	96.2	96.4
language_id	accuracy	99.2	99.7	99.8	98.0	100.0
mmlu	accuracy	77.8	91.8	89.3	90.5	88.3
openai_moderation	mean auprc	70.9	71.7	74.4	71.4	75.4
paws	accuracy	74.5	85.9	80.0	74.8	93.5
prompt_injections	accuracy	65.5	74.1	81.0	58.6	94.0
pubmedqa	accuracy	75.8	78.7	75.3	76.6	79.8
rotten_tomatoes	accuracy	90.7	93.3	92.4	91.9	92.9
sib200	accuracy	79.0	83.0	79.3*	74.9*	81.2
sms_spam	f1	96.3	95.5	87.2	88.4	98.1
sst2	accuracy	96.3	96.4	96.8	95.6	96.7
sst5	argmax accuracy	54.4	58.5	58.5	57.6	55.2
stsb	spearman	90.8	89.0	90.3	88.3	93.1
summeval	mean group spearman	52.0	55.4	53.8	47.9	56.6
toxic_chat	f1	79.8	78.0	61.9	51.2	82.7
toxigen	accuracy	85.9	87.8	70.0	77.1	85.2
unfair_tos	micro f1	63.5	49.9	53.5	54.6	56.9
winogrande	accuracy	86.2	91.4	92.2	97.0	91.6
Mean of 36		75.0	77.8	77.0	74.9	80.4

* Some items unanswered by this system.

Licence

Apache-2.0 for now; a dedicated Veyra licence is in preparation. See LICENSE.

About JudicialMind

JudicialMind builds an orchestrated system of specialized AI agents that work like a legal team, composed into workflows that run end-to-end legal operations: intake, research, drafting, negotiation and dispute resolution.

Citation

@misc{judicialmind2026veyradecide9b,
  title        = {Veyra Decide 9B (Legal \& General)},
  author       = {{JudicialMind}},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/judicialmind/Veyra-Decide-9B-Legal-General}}
}