Qwen3.5-4B-LT-med

LoRA adapter and Clef decision head on unsloth/Qwen3.5-4B, Unsloth's copy of Qwen/Qwen3.5-4B. The base weights are Apache-2.0. This repository does not contain the base weights.

The adapter was trained on 20,000 English decisions from the commercial-ok subset of lighteternal/biodecision-v2-data (Tev1). It was not trained on MEDLINE. The Lithuanian numbers are a transfer evaluation on AIML-VU/MEDLINE-S1-LT, the test split only: 4,857 abstracts and 29,654 decisions.

It was trained for 1.554 of 2 planned epochs. The run stopped at its time limit. The checkpoint figures are in eval_results.json.

Prepared by Linas Petkevičius, Artificial Intelligence Methods Lab, Vilnius University (AIML-VU).

Result on MEDLINE-S1-LT test

Same split for both rows. Accuracy is the fraction of decisions that match the gold option. Record accuracy is the fraction of abstracts on which every question is correct. Both rows use Unsloth's Clef head: one forward pass per abstract. The untouched Qwen row is a new head at random initialization (seed 3407), not Qwen answering as a chatbot.

Model	Accuracy	Correct decisions	Record accuracy	Fully correct abstracts	ECE	Loss
`unsloth/Qwen3.5-4B`, random Clef head	0.548122	16,254 / 29,654	0.012765	62 / 4,857	0.075855	0.798935
This adapter	0.850712	25,227 / 29,654	0.398600	1,936 / 4,857	0.046015	0.396298

Truncation at 2,048 tokens: 57 of 4,857 abstracts, same build for the base and the adapter. Forward-pass time was 1314.8 s for the base and 1535.1 s for the adapter.

By question

Question	Qwen3.5-4B, random Clef head	This adapter
`design`	346 / 1,944	1,627 / 1,944
`humans`	3,635 / 4,857	3,696 / 4,857
`animals`	1,946 / 4,857	4,467 / 4,857
`adverse`	3,904 / 4,857	4,264 / 4,857
`drug_therapy`	2,633 / 4,857	4,166 / 4,857
`disease_area`	230 / 1,014	936 / 1,014
`group_female`	529 / 1,052	796 / 1,052
`group_male`	521 / 1,032	798 / 1,032
`group_older_adults`	477 / 1,012	842 / 1,012
`group_infants`	530 / 1,024	995 / 1,024
`group_adolescents`	499 / 1,088	982 / 1,088
`group_children`	479 / 1,061	993 / 1,061
`group_adults`	525 / 999	665 / 999

By question kind

Kind	Qwen3.5-4B, random Clef head	This adapter
choice	576 / 2,958	2,563 / 2,958
noul	15,678 / 26,696	22,664 / 26,696

Earlier English development split

These numbers are the experiment's own English BioDecision development split, 2,000 decisions, not MEDLINE. One decision per row, so record accuracy equals accuracy.

Split	Accuracy	ECE	Loss
English dev, before fine-tune	0.327500	0.059221	1.359651
English dev, after	0.771500	0.024344	0.587629
English calibration, after the temperature fit	0.783500	0.024743	0.556641

The before row is the random Clef head on the English development file, from qwen/summary.md. Temperatures were fit there and were not refit on MEDLINE.

Load

The base model is downloaded on first use. This directory is the adapter.

from unsloth import FastDecisionModel

model, tokenizer = FastDecisionModel.from_pretrained(
    "AIML-VU/Qwen3.5-4B-LT-med",
    max_seq_length=2048,
    load_in_4bit=True,
)
FastDecisionModel.for_inference(model)
answers = FastDecisionModel.predict(model, tokenizer, state, questions)

state is the Lithuanian title and abstract. questions is the list from MEDLINE-S1-LT. answers maps each question name to answer and probabilities. A local copy of this folder can be passed instead of the repository id.

Training


Base	`unsloth/Qwen3.5-4B`
Data	20,000 English commercial-ok BioDecision v2 rows. Not MEDLINE-S1-LT.
Method	Unsloth `DecisionTrainer`, 4-bit load, LoRA r=16, alpha=16, dropout 0
Steps	Epoch 1.554 of 2. Stopped at the 9900 s time limit. Measured training time 9903.3 s.
Train loss	0.559280
Seed	3407
Context	2,048 tokens. 20,000 training rows.
Temperatures saved with the adapter	1.038977, 0.676812, 1.069501 (choice, score, noul). Folded head temperature 2.090869. Not refit on MEDLINE.
Trainable parameters	64,511,492 of 3,072,856,068

Batch size 2, gradient accumulation 4, learning rate 2e-4, cosine schedule, warmup 100 steps, weight decay 0.01, bf16. Optimizer checkpoints are not in this folder.

Files

File	What it is
`adapter_model.safetensors`, `adapter_config.json`	LoRA on the Qwen language model
`joint_head.safetensors`, `joint_head_config.json`	Clef decision head and its saved temperatures
`unsloth_decision_config.json`	Decision layout, base model, context length
`eval_results.json`	Measured MEDLINE comparison and the training figures
`tokenizer.json`, `tokenizer_config.json`, `chat_template.jinja`, `processor_config.json`	Tokenizer used at training
`LICENSE`, `NOTICE`	Apache-2.0 for this adapter, plus attribution

Licence and attribution

This adapter is Apache-2.0, the licence of the Qwen3.5 base. The Apache text is LICENSE. It does not relicense the base weights, the BioDecision source text, or the MEDLINE abstracts.

Training rows were the commercial-ok subset. Source rows keep their own licences. Non-commercial BioDecision sources were excluded. See the BioDecision v2 card.

MEDLINE-S1-LT translates pritamdeka/MEDLINE-S1 (CC BY 4.0; Pritam Deka, arXiv:2610.02486). Titles and abstracts are machine translations, made with facebook/nllb-200-distilled-1.3B, of MEDLINE/PubMed citation text. Copyright in the underlying English abstracts stays with the publishers or authors. NLM does not own those abstracts and does not endorse this model.

Qwen3.5 is used under its Apache-2.0 licence. Unsloth is the loader and the decision trainer.

Limits

The MEDLINE result is transfer from English BioDecision training, not an in-distribution Lithuanian fine-tune. It is not a substitute for the models trained on MEDLINE-S1-LT. It is not a medical device and it does not give clinical advice.

Citation

@misc{qwen35_4b_lt_med2026,
  title = {Qwen3.5-4B-LT-med},
  author = {Petkevičius, Linas},
  year = {2026},
  howpublished = {\url{https://huggingface.co/AIML-VU/Qwen3.5-4B-LT-med}},
  note = {LoRA adapter and Clef head on unsloth/Qwen3.5-4B.}
}

This project was funded by the European Union (project No S-ITP-26-11) under the agreement with the Research Council of Lithuania (LMTLT).