This model is a QLoRA (Quantized Low-Rank Adaptation) fine-tune of microsoft/Phi-3.5-mini-instruct. It is specifically optimized to act as a domain expert in Egyptian History, providing accurate, concise, and structured answers about ancient monuments, pharaohs, and historical sites.
This adapter was developed as a complete end-to-end MLOps project to demonstrate the ability to train, track, and deploy LLMs.
microsoft/Phi-3.5-mini-instruct (3.8B parameters)The model was fine-tuned on a custom-curated dataset of 200 high-quality English Q&A pairs focusing on Egyptian historical sites (Pyramids of Giza, Karnak, Abu Simbel, Valley of the Kings, etc.).
Because this is a QLoRA adapter, you must load the base model first.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
base_model_id = "microsoft/Phi-3.5-mini-instruct"
adapter_id = "Manar-Abdelgawad/phi-egypt-history-qlora"
# 1. Load in 4-bit (QLoRA configuration)
bnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.float16)
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(base_model_id, quantization_config=bnb_config, device_map="auto")
# 2. Load the adapter
model = PeftModel.from_pretrained(base_model, adapter_id)
# 3. Generate
prompt = "<|user|>\nWho built the Great Pyramid of Giza?<|end|>\n<|assistant|>\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=60)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
checkpoint-78/optimizer.pt 202 MBadapter_model.safetensors 101 MBcheckpoint-78/adapter_model.safetensors 101 MBcheckpoint-78/rng_state.pth 14.6 kBcheckpoint-78/training_args.bin 6 kBtraining_args.bin 6 kBcheckpoint-78/scheduler.pt 1.5 kBmicrosoft/Phi-3.5-mini-instruct