VNLegal-LAL

A Vietnamese legal domain embedding model fine-tuned from mainguyen9/vietlegal-harrier-0.6b (600M params, 1024-dim).

Achieves NDCG@10 = 0.8488 on the Zalo AI Legal Text Retrieval benchmark, setting a new state-of-the-art on Vietnamese legal text retrieval.

Benchmark Results

Evaluated on MTEB ZacLegalTextRetrieval (61.4K corpus documents, 818 test queries) using pytrec_eval (same evaluation library as MTEB).

Key highlights:

• +6.75 points NDCG@10 over previous SOTA (vietlegal-harrier-0.6b): 0.8488 vs 0.7813
• +6.90 points MRR@10: 0.7993 vs 0.7303
• +4.51 points Recall@10: 0.9772 vs 0.9321
• +12.78 points over original Harrier-0.6b (0.8488 vs 0.7210)
• New state-of-the-art on Vietnamese legal text retrieval benchmark

Usage

import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer

model_name = "darklethelong/vnlegal-lal"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name, torch_dtype=torch.float16, device_map="cuda")
model.eval()

QUERY_PREFIX = "Instruct: Given a Vietnamese legal question, retrieve relevant legal passages that answer the question\nQuery: "

def encode(texts, is_query=False, max_len=2048, batch_size=32):
    all_embs = []
    for i in range(0, len(texts), batch_size):
        batch = texts[i:i+batch_size]
        if is_query:
            batch = [QUERY_PREFIX + t for t in batch]
        inputs = tokenizer(batch, padding=True, truncation=True,
                           max_length=max_len, return_tensors="pt").to("cuda")
        with torch.no_grad():
            out = model(**inputs)
        last_idx = inputs["attention_mask"].sum(dim=1) - 1
        embs = out.last_hidden_state[torch.arange(len(batch), device="cuda"), last_idx]
        embs = F.normalize(embs, p=2, dim=1)
        all_embs.append(embs.cpu())
    return torch.cat(all_embs, dim=0)

# Encode
query_embs = encode(["Quy định về hợp đồng lao động"], is_query=True)
doc_embs = encode(["Điều 13. Hợp đồng lao động..."])

# Cosine similarity
similarity = (query_embs @ doc_embs.T).item()

Model Details

• Model Type: Sentence Transformer (decoder-only)
• Base model: mainguyen9/vietlegal-harrier-0.6b
• Maximum Sequence Length: 2048 tokens
• Output Dimensionality: 1024 dimensions
• Pooling: Last-token
• Similarity Function: Cosine Similarity
• Language: Vietnamese
• License: Apache 2.0

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 2048, 'do_lower_case': False}) with Transformer model: Qwen3Model
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_lasttoken': True})
  (2): Normalize()
)

Training

Training Data

• Benchmark dataset: GreenNode/zalo-ai-legal-text-retrieval-vn — 61.4K corpus documents, ~2.4K training queries with relevance judgments

Training Pipeline

Stage 1: BM25 Index Construction
|  Build lexical index over 61.4K corpus documents
|
Stage 2: Iterative ANCE Training (3 rounds)
|  For each round:
|    - Encode corpus + queries with current model
|    - Mine hard negatives (dense + BM25 hybrid)
|    - Filter false negatives (TopK-PercPos)
|    - Train with InfoNCE loss + LoRA
|
Stage 3: Merge & Evaluate
|  Merge LoRA adapter into base model
|  -> Final model (NDCG@10 = 0.8488)

Citation

@misc{vnlegal-lal,
  title={VNLegal-LAL: Vietnamese Legal Domain Embedding Model},
  author={Le The Long},
  year={2025},
  url={https://huggingface.co/darklethelong/vnlegal-lal}
}

Acknowledgements

Special thanks to Ryan Susman and the AIFactory team at CST, Fiserv for granting permission to use compute resources and publish this model.

• Base model: mainguyen9/vietlegal-harrier-0.6b
• Benchmark: Zalo AI Legal Text Retrieval