Ivo Sage

Ivo Sage is a model for long-horizon agentic legal and contract work. It reads the source documents of a matter and produces the requested work product, such as a memo, a redline or a draft agreement, over tens of tool calls. The model uses DeepSeek V4 Flash as its base, post-trained with reinforcement learning on 1,040 training tasks of the Legal Agent Benchmark (LAB), with the benchmark's own agent harness as the environment and an LLM judge's rubric score as the reward. It was built in partnership with River, which provided valuable training infrastructure support.

On the 180 held-out LAB validation tasks it raises the criterion pass rate from 0.824 to 0.915 and the all-criteria pass rate from 7.2% to 15.6%; on Contracts tasks the criterion pass rate rises from 0.701 to 0.913. On General tasks its all-criteria pass rate (15.2%) is comparable to that of Fable 5.1 (15.7%) at 0.4% of the cost per task and exceeds Kimi K3, GPT-6 Astra and GPT-5.6 Sol under the same protocol. On RedlineBench, which is not part of training, it improves the score from 30.8 to 45.7, and on LegalBench it retains the base model's accuracy. The technical report, Ivo Sage: An Open-Source Model for Long-Horizon Agentic Contract Work, is included in this repository.

Model details

• Developed by: Ivo AI
• Base model: deepseek-ai/DeepSeek-V4-Flash-0731, a mixture-of-experts model with 284B total and 13B active parameters
• Trained weights: LoRA, rank 32, alpha 64, on the feed-forward projections (gate_proj, up_proj, down_proj, including the experts) and the output projection (lm_head); attention weights frozen
• Format: LoRA weights in PEFT format: adapter_config.json and two safetensors shards with an index, 12.2 GiB in total
• Training: Reinforcement learning (CISPO) on LAB, 65 optimizer steps
• Context window: 262,144 tokens in training and evaluation
• Language: English

Results

All LAB scores below are measured on the 180 held-out validation tasks (125 General, 55 Contracts): one episode per task in the benchmark's harness, three judge passes per episode, and a task counts as an all-criteria pass only when all three passes find every criterion passed. Gains are paired per task with 95% task-bootstrap intervals.

Harvey LAB, held-out validation tasks

Comparison with frontier models

Twelve other models, among them the top of the Vals AI LAB leaderboard, were run under the same protocol: the same 180 tasks, harness and judge. Cost per task is each model's own token usage at its provider's list price. Sorted by General criterion pass rate.

* Scores exclude tasks the model failed to complete: 3 General and 1 Contracts for each Muse Spark model, 4 General for Fable 5.1, 8 General and 1 Contracts for Sonnet 5.5. † Mean over the tasks with a usage record: one task of the suite, scored zero, has none.

Ivo Sage is on the Pareto frontier of score against cost on both suites and both metrics. Because it samples fewer tokens than the base model, it also costs less per task than the base model.

Other benchmarks

No RedlineBench task, harness or judge is used in training. On LegalBench, Ivo Sage retains the base model's short-form legal reasoning: its score is within 3 points of the base model's in each of the benchmark's five categories.

Training

Data. Tasks come from LAB's General set (1,251 tasks in 24 practice areas) and Contracts set (498 tasks in 14 sub-areas): 1,120 for training and 180 for validation. Contracts tasks that share an agreement family or a source document are kept on the same side of the split. Training draws 16 new tasks per step in a stratified order without repetition; the 65 steps of Ivo Sage use 1,040 of the training tasks (746 General, 294 Contracts), each once. The task lists of both sets are in the training artifacts dataset.

Environment. Each episode runs in the benchmark's own agent loop with its six tools (bash, read, write, edit, glob, grep) in a per-task sandbox, until the model stops calling tools or reaches a limit of 200 turns or 32,768 generated tokens per turn. The benchmark's harness and rubrics are followed strictly, with no retrieval, task-specific prompting or context compaction added. An episode that overflows the context window, is cut off at the per-turn limit or emits a malformed tool call ends at that point and is graded on the files written so far.

Reward. An LLM judge scores each episode's deliverables against the task's rubric in three independent passes. Each pass yields r = 0.8 p_crit + 0.2 p_all, where p_crit is the fraction of criteria passed and p_all is one if every criterion passed; the episode reward is the mean over the three passes.

Optimization. Each step samples 16 tasks and applies one CISPO update, with advantages standardized within each task's group and a small penalty on deliverable length. Table 1 of the technical report lists the full configuration.

Usage

Download the weights:

hf download ivo-ai-labs/Ivo-Sage --local-dir ivo-sage

We suggest serving the model with SGLang, which loads the weights on the base model, e.g., with --tp set to the number of GPUs:

python -m sglang.launch_server \
  --model-path deepseek-ai/DeepSeek-V4-Flash-0731 \
  --trust-remote-code \
  --tp 8 \
  --enable-lora \
  --lora-paths ivo-sage=./ivo-sage

The LoRA weights cover every expert's feed-forward projections and lm_head, stored in two safetensors shards, and SGLang loads them as above. Every evaluation of Ivo Sage in the technical report used thinking mode with the max reasoning effort, temperature 1.0 and top-p 1.0. The base model has no Jinja chat template; the encoding scripts in its repository define the prompt format.

Intended use

Ivo Sage produces legal work product for review by a qualified lawyer. It is trained in English as a tool-using agent: an agent loop with file tools over the documents of a matter.

Evaluation record

The dataset ivo-ai-labs/ivo-sage-training-artifacts holds the task lists of the training and validation sets, every validation episode's transcript, deliverables and judge verdicts, the per-step training records, the other models' scores and the other benchmarks' per-task grades.

Citation

@techreport{ivo2026sage,
  title       = {Ivo Sage: An Open-Source Model for Long-Horizon Agentic Contract Work},
  author      = {{Ivo AI}},
  institution = {Ivo AI},
  year        = {2026},
  month       = sep,
  url         = {https://huggingface.co/ivo-ai-labs/Ivo-Sage}
}