bangla-smollm-135m-cupcake

A small Bangla/Banglish emotion-aware companion language model, fine-tuned from rnnandi/bangla-smollm-135m with QLoRA. It powers the on-device chat companion in the CupCake project, where it is spoken to in mixed Bangla (Banglish) and replies with a leading emotion tag, an optional *action*, and short casual text.

Example:

user:      dosto i'm sleepy now
assistant: [sleepy] *lies down* ghum... goodnight...

Model Details

• Developed by: bayazidhs04
• Model type: Causal LM (LLaMA architecture, SmolLM-135M family)
• Base model: rnnandi/bangla-smollm-135m
• Language(s): Bangla (bn), English (en), and romanized Bangla ("Banglish")
• License: Apache-2.0 (inherited from the base model)
• Finetuned from: rnnandi/bangla-smollm-135m

Architecture

Emotion tags

The assistant prefix encodes the emotion that drives the companion's face/animation in the host app. Observed tags include:

[happy] [sad] [angry] [surprised] [confused] [sleepy] [excited] [neutral] [shy] [bored]

Uses

Direct Use

On-device / edge chat companion for Bangla-speaking (or Banglish-typing) users. Small enough to run on a phone (via llama.cpp GGUF quants) alongside a BLE-connected microcontroller.

Out-of-Scope Use

• Factual question answering, coding, math, or any high-stakes decision support.
• Any safety-critical or medical/legal/financial use.
• Long-form generation — the model is tuned for short, casual, in-character turns.

Training Details

Training Data

bayazidhs04/cupcake-liltii — ~4K emotion-tagged Banglish chat samples (single-turn + multi-turn), split 3600 train / 400 validation.

Sample record:

{"messages": [
  {"role": "user", "content": "hello i had a bad day actually"},
  {"role": "assistant", "content": "[sad] *sits beside you* oh tumi... my sathe thak, sob kichu good hobey..."}
]}

Training Procedure

• Method: QLoRA (PEFT/LoRA on 4-bit base)
• LoRA config: r=8, alpha=16, dropout=0.05, targets q_proj, k_proj, v_proj, o_proj
• Epochs: 3
• Batch size: 4 × gradient accumulation 4
• Learning rate: 2e-4, cosine schedule, 50 warmup steps
• Weight decay: 0.01
• Max seq length: 2048
• Precision: fp16 (fp32 master on fp16 base)
• Hardware: 1× Tesla T4 (16 GB), trained on Kaggle
• Seed: 42

The released model.safetensors here is the LoRA adapter merged back into the fp16 base model.

Files

Why Q4_0 and not k-quants? This model's tensor dimensions (hidden 576, KV projection 192, vocab 180,177) are not divisible by the 256-element blocks that Q2_K/Q3_K/Q4_K need, so llama.cpp silently falls back to higher-precision types for almost every tensor. The resulting "Q2_K" file is ~174 MB at an effective 6.35 bits/weight — larger and worse than a clean Q4_0. Q4_0 (32-element blocks) applies to every tensor here; combined with quantized token embeddings and output tensor it reaches 4.50 bits/weight, i.e. ~127 MB.

Benchmarks

Measured with llama.cpp build b11669 on 4 CPU threads, all models Q4_0, using each model's own chat template. PPL / BPB are on an in-domain Banglish chat corpus (40,645 bytes). tag% = replies starting with an [emotion] tag, action% = replies containing an *action* beat, chars = average reply length, over 25 chat prompts (temperature 0.7, seed 42).

Honest result: this fine-tune is a regression. It scores worse than its own base model (BPB 3.04 → 3.76, PPL 413 → 1006) and tends to ramble in repetitive loops, never emitting the [emotion] format it was trained on. It is kept published for transparency and reproducibility, but for real use prefer liltii-v0.2-qwen2.5-0.5b — 2× larger but ~2.8× better BPB and the only model that follows the persona. BPB (bits per byte) is shown because it is comparable across tokenizers; raw PPL is not.

How to Get Started

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "bayazidhs04/bangla-smollm-135m-cupcake"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)

messages = [{"role": "user", "content": "dosto i'm sleepy now"}]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(prompt, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=64, do_sample=True, temperature=0.8, top_p=0.9)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

llama.cpp

The GGUF carries the chat template, so use conversational mode (-cnv):

llama-cli -m bangla-smollm-135m-v3-q4_0.gguf -p "dosto i'm sleepy now" -cnv -n 64

Bias, Risks, and Limitations

• Trained on a small synthetic/augmented Banglish chat set; outputs are often non-grammatical mixed Bangla/English and should be treated as stylistic, not correct Bangla.
• May reproduce stereotypes or inappropriate slang present in the training data.
• Not safety-aligned: no RLHF/DPO stage. Do not deploy to end users without an output filter, and keep the companion's persona deliberately non-abusive.
• Emotion tags are learned conventions of this dataset, not a reliable affect classifier.

Recommendations

Use the Q4_0 GGUF for local inference, keep temperature around 0.7-0.9, and pair with a host-side safety filter. Always disclose to users that replies are model-generated.

Citation

@misc{bangla-smollm-135m-cupcake,
  title        = {bangla-smollm-135m-cupcake},
  author       = {bayazidhs04},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/bayazidhs04/bangla-smollm-135m-cupcake}},
  note         = {QLoRA fine-tune of rnnandi/bangla-smollm-135m}
}