Sign up

nachiketj14/Girlfriend

🤗 Hugging Face sourcetext-generationapache-2.09.2 GBGGUF✓ Checksums witnessedupdated today
No torrent yet →

Girlfriend - Ministral-3B-Instruct LoRA Finetune

A LoRA finetune of unsloth/Ministral-3-3B-Instruct-2512, trained on my own custom question/answer conversational dataset to build a personal, on-device "companion" chat model. Trained end-to-end on a single Kaggle T4 GPU using Unsloth.

This repo provides three formats of the same finetune so you can pick what fits your use case:

File Format Use case
Girlfriend_3B_lora_adapter LoRA adapter Apply on top of the base model with Unsloth
Girlfriend_3B_F16.gguf Merged FP16 GGUF Full-precision inference
Girlfriend_3B_Q4_K_M.gguf GGUF, Q4_K_M Lightweight on-device inference via llama.cpp, LM Studio, Ollama, KoboldCPP

Model description

  • Base model: unsloth/Ministral-3-3B-Instruct-2512 (text-only Unsloth repack of Mistral AI's Ministral-3 3B Instruct, 2512 release)
  • Finetuning method: LoRA (rank 32) via Unsloth's FastLanguageModel, merged into the base weights for the FP16/GGUF exports
  • Task: single-turn conversational Q&A / persona-style chat
  • Training data: my own dataset of {"question": ..., "answer": ...} pairs, formatted as plain two-turn [user, assistant] chat examples
  • Persona: trained without a system prompt baked into the data - the persona comes entirely from the training examples themselves. You can optionally add a system prompt at inference time to nudge tone further; see Usage below.

Training details

Trained on Kaggle, single T4 GPU (16 GB), using Unsloth's UnslothTrainer / UnslothTrainingArguments, FP16 precision (the T4 has no native BF16 tensor cores).

LoRA configuration

Parameter Value
Rank (r) 32
Alpha 64
Dropout 0.05
Target modules q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Bias none
Gradient checkpointing Unsloth (smart/offloaded)

Optimization

Parameter Value
Learning rate 2e-5
LR scheduler cosine, 3% warmup
Optimizer adamw_8bit
Weight decay 0.01
Max grad norm 0.3
Max steps 60
Per-device train batch 16
Gradient accumulation 2
Effective (global) batch size 32
Max sequence length 4096
Precision FP16
Seed 3407

Label masking

Training used response-only ("completions-only") masking: only the assistant's answer tokens contribute to the loss, while the user's question tokens are masked out (-100). This was implemented via prefix-diffing - tokenizing progressively longer message prefixes and taking the token-id delta across each assistant turn - which works reliably regardless of the exact chat-template markers used.

Usage

The model was trained without a system prompt, so it works fine with none. Optionally, you can nudge the tone/persona at inference time with something like:

You are a loving and caring girlfriend.

Example: GGUF (llama.cpp / LM Studio / Ollama / KoboldCPP)

Download Girlfriend_3B_Q4_K_M.gguf and load it like any other GGUF model.

llama.cpp (CLI):

./llama-cli -m Girlfriend_3B_Q4_K_M.gguf \
  -p "You are a loving and caring girlfriend." \
  -cnv

LM Studio / Ollama / KoboldCPP: import the .gguf file directly, and optionally paste the persona line above into the app's system-prompt field.

Intended use & limitations

  • Intended for personal, on-device conversational / roleplay use, trained on a small, personal dataset.
  • This is a small (3B parameter) model finetuned with LoRA on a narrow domain - it is not a general-purpose instruction-following model and will underperform the base model on tasks outside its training distribution (coding, math, factual QA, long-context reasoning, etc.).
  • Responses reflect a specific conversational persona and were not evaluated for factual accuracy, safety, or bias. Don't rely on it for factual information or advice.
  • As with any small finetuned LLM, expect occasional repetition, hallucination, or inconsistent persona adherence outside the style of the training data.

Files in this repository

  • Girlfriend_3B_lora_adapter - LoRA adapter weights + tokenizer (PEFT-compatible)
  • Girlfriend_3B_F16.gguf - merged base + LoRA weights, converted to GGUF, FP16
  • Girlfriend_3B_Q4_K_M.gguf - same model, quantized to Q4_K_M for lightweight/mobile on-device inference

Acknowledgements