Model Card for Gemma-3-1B Fine-tuned on GSM8k

Summary

This repository contains a LoRA (Low-Rank Adaptation) adapter for the google/gemma-3-1b-it model. It has been fine-tuned using Supervised Fine-Tuning (SFT) on the GSM8k dataset to enhance its ability to solve grade-school math word problems.

The model is specifically trained to generate step-by-step reasoning and provide a final numerical answer enclosed in a \boxed{} environment.

How to Use

This is a PEFT (LoRA) adapter. To use it for inference, you must load the base model (google/gemma-3-1b-it) and then apply the adapter weights from this repository. The recommended approach is to merge the adapter for faster inference.

import torch
from peft import PeftModel
from transformers import AutoTokenizer, AutoModelForCausalLM

# --- Configuration ---
# IMPORTANT: Replace these with the correct base model and your adapter repo ID
base_model_id = "google/gemma-3-1b-it"
adapter_id = "[Your-Username]/[Your-Repo-Name]" # e.g., "tahamajs/gemma-3-1b-it-gsmk8-sft"

# --- Load Model & Tokenizer ---
print("Loading base model...")
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

print("Loading tokenizer...")
tokenizer = AutoTokenizer.from_pretrained(base_model_id)

print(f"Loading LoRA adapter from {adapter_id}...")
model = PeftModel.from_pretrained(base_model, adapter_id)
model = model.merge_and_unload() # Merge for easy and fast inference

print("✅ Model ready for inference.")

# --- Prepare Prompt using the format from training ---
sft_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
Solve the following grade-school math problem. Show your reasoning step-by-step and enclose the final answer in `\\boxed{{}}`.

### Input:
{}

### Response:
{}"""

question = "Natalia sold 48 liters of milk in the morning. In the afternoon, she sold 27 liters less than in the morning. In the evening, she sold 15 liters more than in the afternoon. How many liters of milk did she sell in total?"
prompt = sft_prompt.format(question, "")

# --- Generate Response ---
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, use_cache=True, do_sample=False)
response = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]

print("\n--- Generated Response ---")
# The output starts after the prompt, so we can slice the string to see just the generation
print(response[len(prompt):])

Training Details

Training Data

The model was fine-tuned on the train split of the openai/gsm8k dataset. A 90/10 split was created from the original training data, resulting in 6,725 examples for training and 748 for testing.

The data was preprocessed into an instruction-following format. The loss function was masked to only compute loss on the "Response" part of the text, training the model to generate solutions.

Training Procedure

The model was trained using the SFTTrainer from the TRL library with the following configurations:

Quantization

• Type: 4-bit NF4
• Compute dtype: torch.float32
• Library: bitsandbytes

LoRA Configuration

• Rank (r): 16
• Alpha (lora_alpha): 32
• Dropout (lora_dropout): 0.05
• Target Modules: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]

Training Arguments

• Learning Rate: 2e-4
• Optimizer: paged_adamw_8bit -. Batch Size (per device): 4
• Gradient Accumulation Steps: 4 (Effective batch size of 16)
• Max Sequence Length: 1024
• Warmup Steps: 20
• Logging/Save Steps: 10/50

Evaluation

The model's performance was evaluated on the held-out test set of 748 examples from GSM8k.

Metrics

• Accuracy: Measures the percentage of generated answers where the final numerical value extracted from the \boxed{} block exactly matches the reference answer.
• F1-Macro Score: Provides a balanced measure of performance on the exact-match task.

Results

Fill this table with the final results from your report_df.