GM-LLM (Mistral-24B) — LoRA adapters for cancer-related genetic variant interpretation
LoRA adapters that specialize mistralai/Mistral-Small-3.1-24B-Instruct-2503 for literature-grounded interpretation of cancer-related genetic variants (FT → RAFT). Released with: Lee Y. et al. Staged Domain Specialization of Open-Source Large Language Models for Cancer-Related Genetic Variant Interpretation. Diagnostics (MDPI), in submission, 2026.
Contents
| folder | stage | training data | settings |
|---|---|---|---|
ft_adapter/ |
fine-tuning (FT) | insilicogen/llm-mutation-ft-qa (42,918 cancer-literature QA) |
LoRA r16 / α32 / dropout 0.2, attention + MLP, lr 5e-5, 1 epoch, max length 1024 |
raft_adapter/ |
retrieval-augmented fine-tuning (RAFT) | insilicogen/llm-mutation-variant-qa raft_train (1,438 items) |
LoRA r16 / α32 / dropout 0.2, attention + MLP, lr 5e-5, 6 epochs, max length 4096 |
Base model revision: 68faf511d618ef198fef186659617cfd2eb8e33a. The RAFT adapter was trained on the base model with the FT adapter merged; load them in that order.
Usage
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_id = "mistralai/Mistral-Small-3.1-24B-Instruct-2503"
tok = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "insilicogen/GM-LLM-Mistral-24B-LoRA", subfolder="ft_adapter").merge_and_unload()
model = PeftModel.from_pretrained(model, "insilicogen/GM-LLM-Mistral-24B-LoRA", subfolder="raft_adapter")
# system = the instruction in llm-mutation-variant-qa; user = question + ClinVar annotation block + 5 documents
msgs = [{"role": "system", "content": instruction}, {"role": "user", "content": question + "\n\n" + context}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))
Answers follow the format <body> … </body> <citation>(Reference: doc_N, …)</citation> in Korean.
Results (548-item evaluation set, ClinVar annotation in the input)
| base model | GM-LLM | |
|---|---|---|
| Correctness, Gemma4 judge | 0.498 | 0.956 |
| Correctness, Claude Sonnet 5.5 judge | 0.491 | 0.657 |
| Correctness, real retrieval (BM25, 127 items), Gemma4 judge | — | 0.835 |
Correctness is an LLM-judge score (correct 1 / partial 0.5 / incorrect 0) against reference answers; see the paper for judge settings and confidence intervals.
Intended use and limitations
- Research use. Not a medical device and not for standalone clinical decision-making; outputs must be reviewed by qualified experts.
- Evaluated mainly with gold documents in the context; accuracy drops with real retrieval.
- Outputs depend on the correctness of the input ClinVar annotation; erroneous annotations propagate into answers.
- Use is subject to the license and use policy of the base model (mistralai/Mistral-Small-3.1-24B-Instruct-2503).
Citation
@article{lee2026gmllm,
title = {Staged Domain Specialization of Open-Source Large Language Models for Cancer-Related Genetic Variant Interpretation},
author = {Lee, Younghyun and Park, Juhee and Oh, Seungmi and Lee, Eunbyeol and Lim, Sunghee and Jang, Sehyun and Kong, Jinhwa and Kim, Sangcheol and Kim, Hyeongyong},
journal = {Diagnostics},
year = {2026},
note = {in submission}
}