CodeMate-Qwen 1.5B LoRA

CodeMate-Qwen 1.5B is a lightweight coding assistant fine-tuned from Qwen2.5-Coder-1.5B-Instruct using LoRA/QLoRA with Unsloth.

The goal of this project is to create a small, efficient, domain-focused assistant for Python engineering, frontend development, debugging, refactoring, and production-minded code explanations.

This is a LoRA adapter, not a fully merged standalone model.

---

Project Motivation

Most small coding models can generate code, but they often struggle with structured debugging, practical explanations, and production-oriented guidance.

CodeMate was fine-tuned to respond like a senior engineering assistant by encouraging outputs such as:

• Clear implementation plans
• Self-contained code
• Root-cause debugging
• Practical explanations
• Frontend-focused React/Next.js/Tailwind examples
• Pythonic, readable, type-aware solutions

---

Base Model

Qwen/Qwen2.5-Coder-1.5B-Instruct

---

Training Method

The model was fine-tuned using:

• Unsloth
• QLoRA / 4-bit loading
• PEFT LoRA adapters
• SFTTrainer
• Cosine learning rate schedule
• AdamW 8-bit optimizer

LoRA configuration:

rank: 16
lora_alpha: 32
target_modules:
- q_proj
- k_proj
- v_proj
- o_proj
- gate_proj
- up_proj
- down_proj

---

Usage

Install dependencies:

pip install unsloth transformers peft accelerate bitsandbytes

Load the LoRA adapter:

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="micymike/codemate-qwen-lora",
    max_seq_length=2048,
    load_in_4bit=True,
)

---

Example Output

Prompt:

Write a Python function to safely merge two dictionaries by summing matching numeric values.

Example response:

def safe_merge(dict1: dict, dict2: dict) -> dict:
    """Safely merges two dictionaries by summing values of keys that exist in both."""
    merged_dict = {}

    for key in set(dict1.keys()).union(set(dict2.keys())):
        if key in dict1 and key in dict2:
            merged_dict[key] = dict1[key] + dict2[key]
        elif key in dict1:
            merged_dict[key] = dict1[key]
        else:
            merged_dict[key] = dict2[key]

    return merged_dict

---

Author

Built by Michael Moses / micymike as part of a practical AI engineering project focused on LLM fine-tuning, coding assistants, dataset curation, and deployment.