Bánh mì chuyển ngữ — Gemma 4 E4B Fine-Tuned with Unsloth QLoRA

Competition: The Gemma 4 Good Hackathon on Kaggle
Tracks: Digital Equity & Inclusivity (primary) · Unsloth ($10K) · Future of Education
Framework: Unsloth — 2× faster fine-tuning
Base model: google/gemma-4-e4b-it (4B params, instruction-tuned)

This adapter powers Bánh mì chuyển ngữ ("Trans-Bread"), a macOS application that captures any audio playing on a user's device and renders it as live translated subtitles in a floating, click-through overlay — fully on-device, in 16 languages. Built for the 300M+ people worldwide who live in a country where they don't speak the dominant language.

Highlights

• 99.6% training loss reduction — from 2.916 (baseline) to 0.0115 (final)
• 5 epochs of QLoRA fine-tuning on 10,000 high-quality samples
• Only 2.29% of parameters trained (146.8M / 6.4B) via rank-stabilized LoRA
• ~12 hours total training on a single NVIDIA L4 GPU (24 GB)
• Deployed on-device via Apple MLX 4-bit on Apple Silicon — no cloud, no API keys

Use Case

The fine-tuned model is paired with the mlx-vlm runtime in a native macOS Swift app. ScreenCaptureKit captures system audio, a hysteresis VAD segments utterances, and a single-pass Gemma prompt produces the translated subtitle. End-to-end latency is sub-2 seconds, fully offline.

How to Use

With Unsloth (recommended)

from unsloth import FastModel

model, tokenizer = FastModel.from_pretrained(
    "bradduy/banhmi-gemma4-e4b",
    max_seq_length=2048,
    load_in_4bit=True,
)
FastModel.for_inference(model)

messages = [
    {"role": "user", "content": "Translate to Vietnamese: The doctor will see you now."}
]

inputs = tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
).to("cuda")

outputs = model.generate(
    input_ids=inputs,
    max_new_tokens=128,
    temperature=0.3,
    do_sample=True,
)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))

With Transformers + PEFT

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained(
    "google/gemma-4-e4b-it",
    device_map="auto",
    load_in_4bit=True,
)
model = PeftModel.from_pretrained(base_model, "bradduy/banhmi-gemma4-e4b")
tokenizer = AutoTokenizer.from_pretrained("bradduy/banhmi-gemma4-e4b")

With Ollama / llama.cpp (GGUF)

Two pre-quantized GGUF files ship in this repo for local CPU inference:

# Download the recommended Q4_K_M build
hf download bradduy/banhmi-gemma4-e4b banhmi-gemma4.Q4_K_M.gguf --local-dir .

# Build + run with Ollama
ollama create banhmi-gemma4 -f Modelfile
ollama run banhmi-gemma4

The accompanying Modelfile uses the Gemma 4 chat template and ships a translation-specific system prompt out of the box. To switch to the smaller Q3_K_S variant, change the FROM line in the Modelfile.

Supported Languages (16)

English · Vietnamese · Spanish · Chinese (Simplified) · Japanese · Korean · French · German · Portuguese · Russian · Arabic · Hindi · Indonesian · Thai · Italian · Turkish

Training Details

Configuration

Dataset

• Source: mlabonne/FineTome-100k
• Samples used: 10,000
• Format: Multi-turn chat conversations in Gemma 4 native format (role: "model", not "assistant")
• Masking: Only model responses contribute to loss
• No proprietary, personal, or copyrighted data is used in training.

Hardware

• GPU: NVIDIA L4 (24 GB VRAM)
• RAM: 32 GB
• Training time: ~12 h (with checkpoint resume)
• Peak GPU memory: ~14.8 GB during training

Experiment Journey

We ran 8 systematic experiments to find the optimal configuration:

The multi-epoch discovery

Epoch 1: loss ~0.90  (learning patterns)
Epoch 2: loss ~0.60  (reinforcing)
Epoch 3: loss ~0.30  (deep memorization)
Epoch 4: loss ~0.10  (fine polishing)
Epoch 5: loss ~0.01  (near-perfect fitting)

Other key insights

1. r=64 with RSLoRA is the sweet spot — r=128 converges slower with no multi-epoch benefit.
2. Lower LR (7e-5) stabilizes long training — higher LR (2e-4) destabilizes after epoch 2.
3. train_on_responses_only is essential — masks user/system tokens so the model only learns from responses.
4. Checkpoint every 250 steps — long CUDA runs crash from memory fragmentation; checkpoint-resume solves it.
5. 10k high-quality samples > 20k samples for multi-epoch — quality over quantity.

Reproduce Training

The full Unsloth training pipeline ships with this model repository:

Quick reproduction (defaults = exp08 winning config — no flags needed):

pip install unsloth
python scripts/train.py

Or run from the pinned YAML for an exact match:

python scripts/train.py --lora-rank 64 --use-rslora \
  --learning-rate 7e-5 --num-epochs 5 --max-samples 10000 \
  --grad-accum 8 --weight-decay 0.01 --warmup-steps 50 \
  --save-steps 250 --save-total-limit 3

On-Device Deployment

For the macOS app we use a smaller MLX-quantized variant of the family: unsloth/gemma-4-E2B-it-UD-MLX-4bit, which fits comfortably on Apple Silicon and produces sub-2-second per-utterance latency. The fine-tuned adapter in this repository targets the larger E4B variant for cloud/Ollama inference.

macOS App Source (apps/macos/)

The full Swift source for the Bánh mì chuyển ngữ menu-bar app is included in this repository under apps/macos/:

Quick build:

cd apps/macos
./build.sh
open "$HOME/Applications/Bánh mì chuyển ngữ.app"

Requires Apple Silicon, macOS 13+, Swift 5.9+, and mlx-vlm available to a Python 3 interpreter (point BANHMI_PYTHON at it).

Limitations

• 4B parameter model — larger variants (26B, 31B) would deliver higher translation quality but require more VRAM.
• Training loss ≠ task accuracy — task-level metrics (BLEU, end-to-end latency) are still being measured and will be added.
• Optimized for short utterance translation — not a general-purpose long-form translation model.
• Audio capture is macOS-only — Windows/Linux ports of the app are future work.

License & Data Provenance

• Adapter weights: released under Google's Gemma Terms of Use (inherited from base model).
• Training data: mlabonne/FineTome-100k under its original license.
• No proprietary or personal data used in training.

Acknowledgments

• Google DeepMind for the Gemma 4 model family
• Unsloth for making QLoRA fine-tuning 2× faster and memory-efficient
• Apple MLX team for on-device 4-bit inference
• Kaggle for hosting the Gemma 4 Good Hackathon
• mlabonne for the FineTome-100k dataset

License

Apache 2.0 (same as Gemma 4)