TinyLlama-1.1B-Chat — QLoRA Fine-tuned on Alpaca

A fine-tuned version of TinyLlama/TinyLlama-1.1B-Chat-v1.0 trained on 5,000 samples from the Alpaca dataset using QLoRA (4-bit quantization + LoRA adapters). LoRA weights are fully merged into the base model — drop-in ready for inference with standard transformers, no PEFT required.

TinyLlama-1.1B is already a strong chat-capable base model. This fine-tune further reinforces Alpaca-style instruction following while keeping the model compact enough to run on a single consumer GPU or even CPU.

---

Model Details

---

Usage

from transformers import pipeline

pipe = pipeline("text-generation", model="sarimahsan101/tinyllama-alpaca-qlora")

prompt = "### Instruction:\nExplain what gravity is.\n\n### Response:\n"
output = pipe(
    prompt,
    max_new_tokens=150,
    do_sample=True,
    temperature=0.7,
    repetition_penalty=1.3,
    no_repeat_ngram_size=3,
)
print(output[0]["generated_text"])

Or load manually:

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "sarimahsan101/tinyllama-alpaca-qlora"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)

prompt = "### Instruction:\nWrite a short poem about the ocean.\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens=150,
        do_sample=True,
        temperature=0.7,
        repetition_penalty=1.3,
        no_repeat_ngram_size=3,
        pad_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(out[0], skip_special_tokens=True))

---

Prompt Format

Trained with the standard Alpaca instruction format:

### Instruction:
<your instruction here>

### Response:

With optional input context:

### Instruction:
<your instruction here>

### Input:
<additional context here>

### Response:

---

Example Outputs

Prompt:

### Instruction:
What is machine learning?

### Response:

Response:

Machine learning is a branch of artificial intelligence that enables computers to
learn from data without being explicitly programmed. Instead of writing rules by
hand, you feed examples to an algorithm and it learns patterns on its own...

---

Training Details

Fine-tuned on Google Colab (T4 GPU, ~20–25 min) using:

• transformers
• peft
• trl — SFTTrainer + SFTConfig
• bitsandbytes
• datasets

QLoRA freezes the base model in 4-bit NF4 precision and trains only low-rank adapter matrices injected into the attention layers. Trainable parameters are roughly 0.5% of total model parameters, making fine-tuning feasible on a free T4 GPU.

---

Comparison with OPT-125M Fine-tune

---

Limitations

• Trained on 5K samples for 1 epoch — not exhaustive coverage of all instruction types
• May hallucinate facts on knowledge-intensive questions
• Not safety-aligned — apply your own guardrails for production use
• English only

---

License

TinyLlama is released under the Apache 2.0 license. This fine-tune follows the same terms and is free for personal and commercial use.