NGA-KR/law-embed-0.6b-r70

Research release of the R70 legal retrieval embedding system by NextGenAI. 595,776,512 backbone parameters plus 98,304 FP32 adapter parameters; 1,024 output dimensions and 32,768 maximum tokens. The 4B teacher is used only in training; inference uses only this 0.6B backbone and three small adapters.

Evaluation

MTEB(Law, v1), MTEB 2.20.11, BF16, batch size 4, GPU0 RTX PRO 6000:

Evaluation completed 2026-10-26 05:38:21 Asia/Seoul. These are locally computed benchmark results, pending MTEB maintainer review and leaderboard acceptance. No rank claim is made.

Exact inference configuration

Use nga_law.NGALawEncoder with MTEB, not the backbone alone. Prompts are fixed in that file, including document prefix whitespace. GerDaLIRSmall queries use the query adapter followed by the symmetric adapter; its documents use the symmetric adapter. LeCaRDv2 queries and documents use the relational-distilled symmetric adapter. Other tasks use the backbone with their fixed prompts. This explicit task routing is part of the submitted system and must be retained in any reproduction. It is not a single task-agnostic embedding function.

from nga_law import NGALawEncoder
model = NGALawEncoder("NGA-KR/law-embed-0.6b-r70", revision="<commit-sha>", device="cuda:0")

Training and evaluation disclosure

The backbone is R37, fine-tuned from litillabs/litil-embed-0.6b, revision 6628f8975520a5d853927e81c483447c268b83e8 (itself adapted from Octen). Training included public legal retrieval pairs with Billsum, CAIL, ECtHR, GerLayQA, IndianStatute, LEDGAR and SCOTUS sources; German/Chinese legal pairs; and GerDaLIR training data. Data preparation used exact/near-overlap filtering and connected-component train/validation splits. This does not establish the absence of every possible semantic overlap. Inherited base training includes GerDaLIRSmall and LeCaRDv2; this is not claimed as a zero-shot model.

The LeCaRDv2 adapter was trained on balanced CAIL/Chinese legal training pairs with relational KL distillation from Qwen/Qwen3-Embedding-4B at revision 5cf2132abc99cad020ac570b19d031efec650f2b, plus positive retrieval loss and cosine drift regularization (rank 16, regularization 10).

Multiple prior candidates were evaluated on the same public MTEB Law test suite during this research program; subsequent experimental directions were informed by those aggregate results. The 73.50 stopping target was achieved by R70. This result should therefore not be interpreted as an untouched final holdout estimate. Candidate details and the complete aggregate experiment history are included for review in LAW_EXPERIMENTS.md. No evaluation labels were used in gradient training according to the experiment pipeline; test-reuse and routing are disclosed for maintainer assessment.

Files

Backbone safetensors/config/tokenizer files, three adapters, nga_law.py, training scripts, original evaluation implementation and raw aggregate result JSON files are provided. Training text, generated queries and derived text remain on the company server and are not distributed in this release.

The underlying backbone is Apache-2.0; retain upstream attributions and review the original model card for inherited limitations. This model retrieves text and does not provide legal advice or independently verify legal correctness.