Rumiii/LlamaTron-RS1-Nemesis-1B

Model Overview

LlamaTron RS1 Nemesis is a medical reasoning model produced by fine-tuning meta-llama/Llama-3.2-1B-Instruct on the Medical-Reasoning-SFT-MiniMax-M2.1 dataset using QLoRA. The dataset contains 204,773 clinical reasoning conversations with full chain-of-thought traces covering differential diagnosis, treatment planning, pharmacology, and clinical case analysis.

Despite being a 1 billion parameter model, it handles complex clinical questions with structured and coherent reasoning.

Training Results

Loss decreased consistently across all steps with train and validation loss tracking closely. No overfitting observed.

Dataset

Trained on Medical-Reasoning-SFT-MiniMax-M2.1 released by Maziyar Panahi under the OpenMed initiative.

How to Use

Load the Model

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

model_id = "Rumiii/LlamaTron_RS1_Nemesis_1B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)

messages = [
    {
        "role": "system",
        "content": "You are LlamaTron RS1 Nemesis, a knowledgeable and compassionate medical AI assistant. Provide accurate, evidence-based medical information clearly and helpfully."
    },
    {
        "role": "user",
        "content": "What are the early symptoms of Type 2 Diabetes?"
    },
]

output = pipe(
    messages,
    max_new_tokens=400,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
)

print(output[0]["generated_text"][-1]["content"])

---

Repository

The full training code, merging scripts, and inference interface are available on GitHub: github.com/sufirumii/LlamaTron-RS1-Nemesis-1B

---

Limitations

• This model is intended for research and educational purposes only
• It is not a substitute for professional medical advice, diagnosis, or treatment
• The model was trained with a maximum sequence length of 512 tokens which may limit performance on longer clinical texts
• Always consult a qualified healthcare provider for medical decisions

---

Credits

• Dataset: Maziyar Panahi and the OpenMed initiative for releasing the Medical-Reasoning-SFT-MiniMax-M2.1 dataset under Apache 2.0
• Base Model: Meta AI for releasing Llama-3.2-1B-Instruct
• Libraries: Hugging Face Transformers, PEFT, TRL, BitsAndBytes, Accelerate

---

License

Apache 2.0 — see LICENSE for details.