A Low-Rank Adaptation (LoRA) adapter fine-tuned on caririsynth/Corpus-nordeste-10k to adapt Qwen/Qwen2.5-1.5B to the vocabulary, literary cadence, and regional expressions of Northeastern Brazilian Portuguese (Português Nordestino).
Standard open-source multilingual and Portuguese language models—especially at the compact 1.5B parameter scale—tend to default to generic web-scraped prose, southeastern journalistic templates, or fragmented blog footers when prompted with regional Brazilian Portuguese. In some cases, prompts containing strong regional slang even trigger unintended code-switching into English.
This adapter addresses that domain gap by adapting Qwen/Qwen2.5-1.5B on a curated regional corpus, enabling the model to maintain coherent narrative flow, regional lexicon, and cultural context while remaining lightweight enough for local CPU, mobile, and edge deployment.
Qwen/Qwen2.5-1.5B (Apache 2.0)caririsynth/Corpus-nordeste-10k (train split, 10,555 curated records)SFTTrainer)r): 16q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_projmax_seq_length): 10242e-4 with cosine scheduler and 30 warmup stepsadamw_8bit (weight decay: 0.01)fp16 compute)The model was trained on caririsynth/Corpus-nordeste-10k, which combines:
Both models were evaluated side-by-side using identical generation parameters (seed=42, temperature=0.65, top_p=0.9, repetition_penalty=1.15, max_new_tokens=100):
| Prompt | Base Qwen/Qwen2.5-1.5B |
Qwen2.5-1.5B-Nordeste-LoRA |
|---|---|---|
| "E aí, macho, tu vai pro forró hoje ou vai ficar de" | "...fora da festa? - Blog do Zico / Home / Sem categoria / Publicado por: Zico em: 10/28/2016 às 4:37 pm..." (collapses into scraped blog metadata) | "...plantão no quarteirão? Pois vá logo que o forró chega devagar quase parando!" |
| "O cabra chegou na bodega todo aperreado, pedindo" | "...uma garrafa de vinho. (...) E eu fiquei com pena porque sabia que o cara ia ter feito tudo errado. Could you please" (drifts off-topic and leaks English) | "...buchada e canjica geladinha. Todo mundo riu, que no sertão até a pressa tem bom humor." |
| "Vixe Maria, o sol hoje tá de rachar mamona, bora ali" | "...na praça do barão de Monte Alto tomar um café e conversar com as galinhas? (...) Postado no blog..." (loses coherence and shifts tone) | "...na praça da matriz pegar um café coado e esperar que ele melhore. E a gente começa essa jornada com força no pé..." |
transformers + peftimport torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_id = "Qwen/Qwen2.5-1.5B"
adapter_id = "caririsynth/Qwen2.5-1.5B-Nordeste-LoRA"
tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(
base_id,
torch_dtype=torch.float16,
device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter_id)
prompt = "O cabra chegou na bodega todo aperreado, pedindo"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=100,
temperature=0.65,
top_p=0.9,
repetition_penalty=1.15,
do_sample=True,
pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Qwen/Qwen2.5-1.5B