LFM2.5-Encoder-350M hallucination detector

A span-level hallucination detector built on LiquidAI/LFM2.5-Encoder-350M: the bidirectional LFM2.5 backbone with a linear token-classification head, fine-tuned to mark the character spans of an answer that are not supported by the given context. Part of the LettuceDetect project.

Trained on the LettuceDetect unified benchmark: coding-agent answers (SWE-bench-derived), developer tool output, structured documents (ACL papers, READMEs, Wikipedia markdown), RAGTruth, and 14-language PsiloQA.

Paper: Beyond Document Grounding: Span-Level Hallucination Detection over Code, Tool Output, and Documents (GroundLM 2026 workshop at EMNLP 2026) · arXiv:2607.00895

Results

Character-level span metrics on the LettuceDetect unified test split (n=10,698):

Comparison

Span-F1 by source against the other LettuceDetect detectors on the same test split:

On multilingual example-level detection it matches the mmBERT encoder (PsiloQA example-F1 0.945 vs 0.943). On code-agent answers it remains far above general-purpose LLM judges at a fraction of their size (Nemotron-3-Ultra-550B 0.216, gpt-oss-120b 0.212 span-F1; HHEM-2.1 / Lynx-8B / Granite-Guardian / MiniCheck ≈ chance).

Usage

# pip install lettucedetect
from lettucedetect.models.inference import HallucinationDetector

detector = HallucinationDetector(
    method="transformer",
    model_path="KRLabsOrg/LFM2.5-Encoder-350M-hallucination-detector",
    trust_remote_code=True,
)

predictions = detector.predict(
    context=["The Eiffel Tower is 330 metres tall and stands in Paris, France."],
    question="How tall is the Eiffel Tower and where is it?",
    answer="The Eiffel Tower is 330 metres tall and stands in Berlin.",
    output_format="spans",
)
print(predictions)
# [{'start': 49, 'end': 56, 'confidence': 0.92, 'text': ' Berlin'}]

Token-level classification without the LettuceDetect wrapper:

import torch
from transformers import AutoTokenizer, AutoModelForTokenClassification

repo = "KRLabsOrg/LFM2.5-Encoder-350M-hallucination-detector"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForTokenClassification.from_pretrained(repo, trust_remote_code=True).eval()

enc = tokenizer("context text ... answer text", return_tensors="pt")
with torch.no_grad():
    labels = model(**enc).logits.argmax(-1)[0]   # 0 = supported, 1 = hallucinated

Training

• Backbone: LiquidAI/LFM2.5-Encoder-350M (bidirectional), linear head, dropout 0.1
• 3 epochs on the unified train split (66,368 samples), input [question, context, answer], answer tokens labeled supported/hallucinated, max length 8,192
• Token-level validation F1 0.635, test F1 0.609

Citation

@inproceedings{kovacs-etal-2026-beyond,
    title = "Beyond Document Grounding: Span-Level Hallucination Detection over Code, Tool Output, and Documents",
    author = "Kov{\'a}cs, {\'A}d{\'a}m  and
      He, Bowei  and
      Liu, Xue  and
      Boros, Istv{\'a}n  and
      T{\'o}th, Szilveszter  and
      Recski, Gabor",
    editor = "Wang, Yimu  and
      Choi, Yee Man  and
      Wu, Di  and
      Ouyang, Siqi  and
      Azadani, Mozhgan Nasr  and
      Fung, Yi R. (May)",
    booktitle = "Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently ({G}round{LM} 2026)",
    month = oct,
    year = "2026",
    address = "Budapest, Hungary",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.groundlm-1.44/",
    pages = "566--577",
    ISBN = "979-8-89176-478-1"
}

Also on arXiv: 2607.00895.