laya-multilingual-lt-med

Lithuanian medical decision model. It is a fine-tune of convaiinnovations/laya-multilingual on AIML-VU/MEDLINE-S1-LT.

Give it a Lithuanian title and abstract (state) and the typed questions from that dataset. In one forward pass it returns a probability for every option. It does not generate text.

Load it with the laya package, not with AutoModelForSequenceClassification. The weights are a merged Laya checkpoint: model.safetensors, encoder/, tokenizer/, and rl_agent_config.json.

This is a research model. It is not a medical device and it is not for clinical decisions.

Result on the held-out test split

The comparison is the official MEDLINE-S1-LT test split: 4,857 abstracts, 29,654 decisions. The same questions and the same gold labels were used before and after fine-tuning.

Before is the released convaiinnovations/laya-multilingual checkpoint, scored as a decision model. It is not a randomly initialized head. After is this checkpoint, with temperatures fitted on the calibration split only. Temperature changes calibration and loss. It does not change the argmax, so the accuracy gain is from the fine-tune.

Record accuracy is the stricter number: an abstract counts only when every question on it is right. That is the "improved records" result. It moves from 151 abstracts (3.11%) to 3,048 abstracts (62.75%).

The training log stored rates rounded to six decimals, not the integer counts. Each count above is the only integer whose ratio rounds back to the logged rate.

No per-question accuracy was stored, so this card does not break the gain down by design, disease_area, or the yes/no questions. eval_results.json is the same report in JSON, without any local path.

After fine-tuning only

Development and calibration were not scored before training. These rows are not a before/after comparison.

The saved weights are the best development-loss state kept during training (development loss 0.202145). Temperatures were fitted after that, on calibration data that was not used to choose the weights.

What the test split contains

These are question counts in the dataset, not model scores.

Option keys stay in English (systematic_review, Neoplasms, true, false). Instructions and option descriptions are Lithuanian. Gold labels come from NLM MeSH and publication types, not from a language model. The dataset card describes the rules.

Use

pip install laya

import laya

agent = laya.load("AIML-VU/laya-multilingual-lt-med")

state = (
    "Atsitiktinių imčių kontroliuojamajame tyrime 240 suaugusių pacientų "
    "gavo tiriamąjį vaistą arba placebą. Vertintas nepageidaujamas poveikis."
)
questions = {
    "design": {
        "type": "choice",
        "instructions": "Koks tai tyrimo ar straipsnio tipas?",
        "criteria": {
            "systematic_review": "ankstesnių tyrimų sisteminė apžvalga arba metaanalizė",
            "randomized_trial": "atsitiktinių imčių kontroliuojamasis tyrimas",
            "other_trial": "klinikinis tyrimas be atsitiktinių imčių",
            "case_report": "vieno ar kelių pacientų atvejo aprašymas",
            "observational": "stebimasis tyrimas (kohortinis, atvejo ir kontrolės arba skerspjūvio)",
            "narrative_review": "naratyvinės apžvalgos straipsnis",
        },
    },
    "humans": {
        "type": "noul",
        "instructions": "Ar šiame tyrime buvo tiriami žmonės (pacientai ar dalyviai)?",
        "criteria": {
            "true": "Teiginys teisingas arba atsakymas yra taip.",
            "false": "Teiginys klaidingas arba atsakymas yra ne.",
        },
    },
}

result = agent.predict(state, questions)
print(result["answers"]["design"]["choice"])
print(result["answers"]["design"]["probabilities"])
print(result["answers"]["humans"]["noul"])  # P(true); at least 0.5 means yes

A local copy of this folder loads the same way:

agent = laya.load("laya-multilingual-lt-med")

state may be a string or a JSON object. questions maps an id to one of choice, score, or noul, which is the Laya API. This fine-tune saw only choice and noul. A yes/no answer is answers[name]["noul"], the probability of true, not a boolean. A choice answer is answers[name]["choice"] plus probabilities.

The checkpoint stores max_len 2048 and head_max_len 256. Training and the numbers above used that limit. The mmBERT encoder can hold 8,192 positions, but this fine-tune was not evaluated at a longer limit. Fifteen of 29,654 test decisions and 114 of 236,720 training decisions were truncated at 2,048 tokens.

If laya.load hangs while TensorFlow is installed, the original Laya card's workaround is to run with USE_TF=0.

Training

Fitted temperatures, applied at prediction time:

The released Laya checkpoint ships temperatures [1.0, 1.0, 1.0]. The original model card says that checkpoint is systematically over-confident and that a temperature refit is required before the probabilities are trusted. The test ECE here, 0.343504 before and 0.003805 after, is consistent with that warning and with the refit on MEDLINE-S1-LT calibration data.

Original model

This repository does not replace convaiinnovations/laya-multilingual. Use that checkpoint for the general multilingual decision model. Use this one for the Lithuanian MEDLINE questions above. Fine-tuning was measured only on MEDLINE-S1-LT. It was not re-measured on the original Laya suites, so this card does not claim that those results still hold.

Laya is a non-autoregressive System 1 decision model from Convai Innovations. A state and typed questions go in; option probabilities come out of one forward pass. There is no generated string to parse. The multilingual checkpoint is the member of the family intended for languages other than English.

The original multilingual card reports its own MASSIVE, XNLI, speed, and long-document measurements, and these limits of the released checkpoint:

• it ships uncalibrated, and confidence stays high when accuracy does not
• it is weaker on English than convaiinnovations/laya
• zero-shot typed decisions are near chance until the model is fine-tuned
• choice questions should stay under about 20 options, because the options share a 256-token head budget
• score questions are the weakest type and have a measured position bias (laya issue 131)
• noul can under-report true on the released checkpoint (laya issue 156)

Those statements describe the original checkpoint. They are not measurements of this fine-tune, apart from the calibration result in the table above.

• Docs: https://nandhakishorm.github.io/laya/
• Family: https://huggingface.co/convaiinnovations/laya
• Code: https://github.com/NandhaKishorM/laya
• Package: https://pypi.org/project/laya/
• Demo: https://huggingface.co/spaces/convaiinnovations/laya-demo
• Encoder: https://huggingface.co/jhu-clsp/mmBERT-base
• Encoder paper: https://arxiv.org/abs/2509.06888

The original model is Apache 2.0, Convai Innovations. LICENSE in this folder is that license. NOTICE names the upstream works.

Data

Train, development, calibration, and test text are the Lithuanian MEDLINE-S1 translation:

https://huggingface.co/datasets/AIML-VU/MEDLINE-S1-LT

The parent English dataset is pritamdeka/MEDLINE-S1, introduced in Pritam Deka, From Retrieval to Typed Decisions: Calibrated System One Models from Biomedical Sentence Encoders, arXiv:2610.02486.

Titles and abstracts were translated with facebook/nllb-200-distilled-1.3B (eng_Latn to lit_Latn, revision 7be3e24664b38ce1cac29b8aeed6911aa0cf0576). Question ids, option keys, and gold labels are unchanged. 1,239 abstracts were dropped because the translation collapsed into repetition.

Licence

The weights are Apache 2.0, because they derive from convaiinnovations/laya-multilingual. Keep LICENSE and NOTICE with any redistribution. The encoder ancestry is jhu-clsp/mmBERT-base, whose card declares MIT.

The training text is not Apache. AIML-VU/MEDLINE-S1-LT offers its package under CC BY 4.0 and asks for attribution to Pritam Deka, arXiv:2610.02486, and to the NLLB translation model. Two limits from that card still apply:

• NLM does not own the abstracts. Copyright in the English text remains with the publishers or authors. NLM's terms apply to that text and to the translations, and NLM does not endorse this model.
• facebook/nllb-200-distilled-1.3B is CC BY-NC 4.0. Its card says the translation model is for research, not for production, medical text, or document translation.

Limits

• Measured on Lithuanian MEDLINE-S1-LT only. Not a general Lithuanian medical chatbot, and not a substitute for the original multilingual checkpoint.
• Not a medical device. Do not use the probabilities as a diagnosis, a screening result, or a treatment recommendation.
• score questions were not in the training data. Their temperature is still 1.0.
• The act/escalate head is part of the Laya output and was not the reported metric.
• Fifteen test decisions were truncated at 2,048 tokens.

Files

laya.load needs the weights, rl_agent_config.json, encoder/, and tokenizer/. There is no trainer/ directory and none should be added.

Citation

@misc{petkevicius2026laya-lt-med,
  title        = {laya-multilingual-lt-med: a Lithuanian MEDLINE decision model},
  author       = {Petkevi{\v{c}}ius, Linas},
  year         = {2026},
  howpublished = {Artificial Intelligence Methods Lab, Vilnius University},
  url          = {https://huggingface.co/AIML-VU/laya-multilingual-lt-med}
}

@misc{deka2026retrievaltypeddecisionscalibrated,
  title         = {From Retrieval to Typed Decisions: Calibrated System One Models from Biomedical Sentence Encoders},
  author        = {Pritam Deka},
  year          = {2026},
  eprint        = {2610.02486},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2610.02486}
}

@article{nllbteam2022language,
  title   = {No Language Left Behind: Scaling Human-Centered Machine Translation},
  author  = {{NLLB Team}},
  journal = {arXiv preprint arXiv:2207.04672},
  year    = {2022},
  url     = {https://arxiv.org/abs/2207.04672}
}

Also credit Convai Innovations, convaiinnovations/laya-multilingual, and the mmBERT encoder, https://arxiv.org/abs/2509.06888.

This project was funded by the European Union (project No S-ITP-26-11) under the agreement with the Research Council of Lithuania (LMTLT).