Qwen3-14B fine-tuned for STR DNA family analysis
- Developed by: haninmb
- License: apache-2.0
- Finetuned from model: unsloth/Qwen3-14B-unsloth-bnb-4bit (Qwen3-14B)
- Weights in this repo: LoRA merged into the base model, 16-bit
The model reads an STR profile (father, mother and child alleles per marker) and returns an analysis of Mendelian inheritance: which markers mismatch, the mutation type and direction, the likely source parent, and a biological / not-biological classification. The answer is written as a short Analysis: section followed by a Final Response: JSON object.
This qwen3 model was trained 2x faster with Unsloth and Huggingface's TRL library.
Training summary
| Method | QLoRA supervised fine-tuning (4-bit base, LoRA adapters), 1 epoch |
| Total training time | 118,618 s (about 32 h 57 min) wall-clock |
| Peak VRAM (reserved) | about 13 GB on a single GPU |
| Hardware | 1 GPU with 32 GB VRAM |
| Throughput | 0.78 samples/s, 0.196 steps/s (about 5.1 s per optimizer step) |
| Optimizer steps | 23,203 (effective batch size 4) |
| Training examples | 92,811 |
| Held-out examples | 938 (1% split, seed 42). Not used for evaluation during training. |
| Final training loss | about 0.068 (running mean over the whole run: 0.072) |
| Checkpoints | every 400 steps, last 3 kept |
Loss fell from about 1.27 at step 10 to about 0.07 by step 2,000 and stayed between 0.066 and 0.070 for the rest of the run. The loss is computed over the whole sequence, including the long system prompt, not only the assistant answer, so it is not comparable with response-only losses.
Training data
Dataset: haninmb/all_merged_families (updates_prompts_all_merged_families_10_4v.jsonl), about 93.7k examples. Each example is a system prompt describing the inheritance rules and output format, a user message with the family STR profile, and an assistant answer. The original analysis and final assistant turns were merged into one answer (Analysis: followed by Final Response:), and the developer role was mapped to system.
Training procedure
| Hyperparameter | Value |
|---|---|
| Base model | Qwen3-14B, loaded in 4-bit |
| LoRA rank / alpha / dropout | 32 / 32 / 0 |
| LoRA target modules | q, k, v, o, gate, up, down projections |
| Max sequence length | 8,048 tokens |
| Per-device batch size | 1 |
| Gradient accumulation | 4 |
| Learning rate | 2e-4, linear decay |
| Warmup steps | 500 |
| Weight decay | 0.001 |
| Optimizer | AdamW 8-bit |
| Precision | bf16 |
| Gradient checkpointing | Unsloth |
| Seed | 3407 |
Software: transformers 4.57.1, TRL 0.22.2, Unsloth 2026.10.11, PyTorch 2.11 (CUDA 12.8).
The training code is in training/ and the TensorBoard logs are in tensorboard/. The run can be resumed from its latest checkpoint after an interruption.
Usage
The 16-bit weights are about 30 GB. On a GPU with less than 40 GB of VRAM, load the model in 4-bit (for example with bitsandbytes or Unsloth, which needs about 10 GB for weights).
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
"haninmb/train_qewn3_final", max_seq_length=16384, load_in_4bit=True
)
FastLanguageModel.for_inference(model)
messages = [
{"role": "system", "content": "<the STR analysis system prompt used in training>"},
{"role": "user", "content": "Input family DNA profile:\nCSF1PO_Father: 10, 12 | Mother: 13, 14 | Child: 11, 13\n..."},
]
inputs = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to("cuda")
out = model.generate(**inputs, max_new_tokens=4048)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Use the system role for the instructions, as in training. Inference on a single consumer GPU in 4-bit runs at roughly 60 to 80 tokens per second, and a typical answer is 300 to 900 tokens.
Repository contents
- Model weights and tokenizer (merged, 16-bit)
training/: training script and launchertensorboard/: TensorBoard event logs for the full run