Veyra Decide 0.6B (Legal & General)

Veyra Decide 0.6B is a legal decision model with general capabilities. It answers typed questions about a document (multiple choice, yes/no, ordinal score) with calibrated probabilities.

Use

from transformers import AutoModel

model = AutoModel.from_pretrained("judicialmind/Veyra-Decide-0.6B-Legal-General", trust_remote_code=True)
result = model.system_one(
    state="Recipient shall not disclose Confidential Information to any third party without prior written consent.",
    questions={
        "third_party_sharing": {
            "type": "noul",
            "instructions": "Does the clause allow sharing confidential information with third parties?",
            "criteria": {"false": "No, sharing is not allowed.", "true": "Yes, sharing is allowed."},
        }
    },
)
print(result["answers"]["third_party_sharing"])

Requires transformers>=5.17, torch and safetensors. Maximum input: 16,384 tokens.

Results

Results were measured with inputs up to 8,192 tokens.

Legal (accuracy, %)

Benchmark	Questions	Veyra Decide 0.6B	Jev	Clef	Clef-flash	Perplexity
ContractNLI	2,091	86.7	77.4†	87.8	88.7	80.7
CUAD	3,895	65.3	89.8	91.1	87.8	92.2
CUAD, contracts within 8K tokens	2,706	94.0	91.5	92.6	89.7	93.5
LEDGAR	3,000	76.7	74.7	73.6	72.2	80.6
CaseHOLD	3,600	76.2	77.6	75.9	69.5	86.7
UNFAIR-ToS	12,856	97.3	97.0	97.4	97.9	98.7

† Published Jev-Benchmark answers.

General (accuracy, %)

Benchmark	Questions	Veyra Decide 0.6B	Jev	Clef	Clef-flash	Perplexity
BoolQ	3,270	81.8	92.0	91.3	91.0	92.8
MNLI	3,000	83.0	87.2	89.6	89.1	92.8
MNLI mismatched	2,996	83.8	86.8	89.3	89.9	92.8
ARC-Challenge	1,172	61.5	97.7	97.7	98.1	96.8
ARC-Easy	2,376	82.6	99.4	99.2	99.6	99.4

Long legal documents (accuracy, %)

Benchmark	Questions	Veyra Decide 0.6B	Jev	Clef	Clef-flash	Perplexity
LexGLUE ECtHR-A	10,000	86.2	92.7	94.5	93.0	93.4
LexGLUE ECtHR-B	10,000	87.3	91.9	94.3	93.4	94.3
LexGLUE SCOTUS	1,317	22.6	71.6	71.1	67.6	78.7
LexGLUE SCOTUS, opinions within 8K tokens	483	61.7	68.7	67.9	65.8	74.5
MAUD	6,651	54.5	64.4	64.0	60.9	66.7
JevBench	1,938	81.7	90.5	90.5	88.1	92.0

CUAD clause retrieval over contract paragraphs (mean AUPR, 40 clause types)

Veyra Decide 0.6B	Jev	Clef	Clef-flash	Perplexity
0.251	0.534	0.521	0.466	0.554

LegalBench (mean task accuracy, %)

Task group	Tasks	Veyra Decide 0.6B	Jev	Clef	Clef-flash	Perplexity
CUAD clauses	38	84.1	84.4	85.4	83.0	85.1
MAUD	34	61.9	79.3	79.2	72.1	76.4
ContractNLI	14	91.6	92.3	94.5	92.6	93.1
Learned Hands	16	78.2	82.5	88.2	86.8	88.6
Supply chain	10	79.3	77.4	76.9	80.9	77.6
OPP-115 privacy	9	76.9	83.4	83.9	80.8	83.4
Diversity	6	47.3	92.2	82.7	74.1	94.1
Other	27	64.1	76.9	75.8	70.7	79.5
All tasks	154	73.6	82.3	82.7	79.1	83.1

Robustness (accuracy, %)

Test	Veyra Decide 0.6B	Jev	Clef	Clef-flash	Perplexity
BoolQ, negated question	78.6	90.4	83.1	53.5	84.8
BoolQ, swapped yes/no descriptions	82.1	9.8	91.0	91.2	92.9
CUAD, negated question (descriptions rewritten to match)	65.3	90.9	88.7	87.1	90.0
CUAD, swapped yes/no descriptions	65.3	11.3	90.9	88.3	92.4
UNFAIR-ToS, negated question (descriptions rewritten to match)	98.7	97.0	97.7	97.2	98.4
UNFAIR-ToS, swapped yes/no descriptions	97.7	2.5	97.5	97.2	98.6
ContractNLI, random option ids	86.8	77.4	87.8	87.9	80.6
ContractNLI, shuffled options	86.8	78.2	87.8	88.7	81.0
CaseHOLD, random option ids	76.0	77.9	75.2	69.4	86.4
CaseHOLD, shuffled options	75.8	76.8	75.9	69.5	86.3
MNLI, random option ids	83.1	86.9	89.0	89.4	93.0
MNLI, shuffled options	83.3	87.3	89.6	89.1	92.9
ARC-Challenge, random option ids	60.4	97.4	97.8	98.0	96.8
ARC-Challenge, shuffled options	60.8	97.8	97.7	98.1	97.3

Decision Index (13 public benchmarks)

Benchmark	Metric	Veyra Decide 0.6B	Jev	Clef	Clef-flash	Perplexity
BANKING77	F1	73.3	79.7	94.1	90.8	80.6
CLINC150	F1	79.0	89.3	97.4	66.9	91.5
ContractNLI	F1	74.9	71.7	81.3	84.5	78.4
ANLI	F1	32.9	74.8	70.0	59.1	74.3
MMLU	accuracy	43.8	91.7	90.2	91.9	87.7
ARC-Easy	accuracy	81.9	99.3	99.0	99.5	99.3
ARC-Challenge	accuracy	59.2	97.8	97.8	98.2	97.0
WinoGrande	accuracy	58.3	92.0	93.5	97.5	92.4
HellaSwag	accuracy	58.2	94.5	98.2	98.6	93.2
NLI4CT	F1	60.5	84.1	83.0	78.6	84.9
MMLU-Pro	accuracy	19.9	82.7	66.0	65.2	71.0
BBH	accuracy	37.9	92.9	73.6	68.9	84.0
RAGTruth	F1	1.4	76.5	79.5	35.6	83.1
Mean of 13		52.4	86.7	86.4	79.6	86.0

Outside-system figures: public Decision Index leaderboard.

ContractNLI against Jev-Benchmark systems (2,091 judgments, accuracy, %)

System	Its accuracy	Veyra Decide 0.6B
Gemini Pro	83.2	86.7
Terra	82.6	86.7
Gemini Flash-Lite	82.5	86.7
Claude Sonnet 5	82.4	86.7
Astra	81.8	86.7
Claude Haiku 4.5	79.4	86.7
Qwen 9B	79.1	86.7
Luna	79.0	86.7
Jev	77.4	86.7
Qwen 4B	74.2	86.7

Outside-system figures: published Jev-Benchmark answers.

Jev harness tasks (Jev's scorer, primary metric x100)

Task	Metric	Veyra Decide 0.6B	Jev	Clef	Clef-flash	Perplexity
afrixnli	accuracy	46.4	62.8	53.4	49.2	70.8
ag_news	accuracy	88.4	87.5	89.3	89.3	89.8
agb_de	f1	4.1	20.4	17.9	7.0	23.5
anli	accuracy	34.1	74.3	62.1	57.1	72.5
arc	accuracy	71.2	98.5	98.6	99.2	98.0
art	accuracy	60.4	83.9	88.3	84.3	88.2
banking77	accuracy	76.3	79.2	94.0	95.2	82.0
belebele	accuracy	42.0	87.5	82.3	73.5	85.7
bigbench	accuracy	46.7*	83.7*	77.5*	72.2*	83.6
boolq	accuracy	78.5	91.0	91.2	91.0	92.3
ceval	accuracy	41.5	84.5	80.0	77.7	84.2
clinc150	accuracy	79.1	89.8	96.7	98.3	90.8
commonsense_qa	accuracy	62.4	88.2	89.4	92.9	88.6
emotion	accuracy	57.0	58.5	59.3	58.1	61.3
financial_phrasebank	accuracy	75.4	73.0	72.6	74.5	85.3
go_emotions	macro f1	27.4	24.1	46.5	50.3	35.3
hellaswag	accuracy	61.3	95.5	98.4	98.8	94.2
helpsteer2	mean spearman	21.1	41.2	43.5	38.6	50.8
imdb	accuracy	91.5	96.2	96.8	96.2	96.4
language_id	accuracy	7.3	99.7	99.8	98.0	100.0
mmlu	accuracy	44.0	91.8	89.3	90.5	88.3
openai_moderation	mean auprc	43.9	71.7	74.4	71.4	75.4
paws	accuracy	45.9	85.9	80.0	74.8	93.5
prompt_injections	accuracy	57.8	74.1	81.0	58.6	94.0
pubmedqa	accuracy	52.8	78.7	75.3	76.6	79.8
rotten_tomatoes	accuracy	83.7	93.3	92.4	91.9	92.9
sib200	accuracy	55.5	83.0	79.3*	74.9*	81.2
sms_spam	f1	5.9	95.5	87.2	88.4	98.1
sst2	accuracy	86.7	96.4	96.8	95.6	96.7
sst5	argmax accuracy	41.6	58.5	58.5	57.6	55.2
stsb	spearman	87.0	89.0	90.3	88.3	93.1
summeval	mean group spearman	27.9	55.4	53.8	47.9	56.6
toxic_chat	f1	58.7	78.0	61.9	51.2	82.7
toxigen	accuracy	66.1	87.8	70.0	77.1	85.2
unfair_tos	micro f1	37.3	49.9	53.5	54.6	56.9
winogrande	accuracy	58.5	91.4	92.2	97.0	91.6
Mean of 36		53.5	77.8	77.0	74.9	80.4

* Some items unanswered by this system.

Licence

Apache-2.0 for now; a dedicated Veyra licence is in preparation. See LICENSE.

About JudicialMind

JudicialMind builds an orchestrated system of specialized AI agents that work like a legal team, composed into workflows that run end-to-end legal operations: intake, research, drafting, negotiation and dispute resolution.

Citation

@misc{judicialmind2026veyradecide06b,
  title        = {Veyra Decide 0.6B (Legal \& General)},
  author       = {{JudicialMind}},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/judicialmind/Veyra-Decide-0.6B-Legal-General}}
}