BERT emotion LoRA adapter (learning repo)

This is not a standalone model.

It is the PEFT LoRA adapter trained on top of google-bert/bert-base-uncased for 6-class English emotion classification (dair-ai/emotion).

For drop-in inference (pipeline), use the merged repo instead:

Shankarblr/shankar-bert-emotion-en

Use this repo when you want to:

• see how a LoRA artifact is packaged (~few MB, not 438 MB)
• load adapters onto the frozen base with PeftModel
• keep training / swap adapters without shipping full BERT again

What is in here

Upload only these files (no checkpoint-* folders):

Leave out optimizer.pt, scheduler.pt, rng_state.pth, and checkpoint-350 … checkpoint-1400. Those are Trainer resume points, not Hub inference files.

Labels

sadness (0), joy (1), love (2), anger (3), fear (4), surprise (5)

Load the adapter

You must start from the same base architecture the adapter was trained on, including num_labels=6.

from transformers import AutoTokenizer, AutoModelForSequenceClassification
from peft import PeftModel

BASE = "google-bert/bert-base-uncased"
ADAPTER = "Shankarblr/bert-emotion-lora-adapter"

tokenizer = AutoTokenizer.from_pretrained(ADAPTER)  # or BASE
base = AutoModelForSequenceClassification.from_pretrained(
    BASE,
    num_labels=6,
    id2label={
        0: "sadness",
        1: "joy",
        2: "love",
        3: "anger",
        4: "fear",
        5: "surprise",
    },
    label2id={
        "sadness": 0,
        "joy": 1,
        "love": 2,
        "anger": 3,
        "fear": 4,
        "surprise": 5,
    },
)
model = PeftModel.from_pretrained(base, ADAPTER)
model.eval()

text = "I am worried with the current job market"
inputs = tokenizer(text, return_tensors="pt")
pred_id = model(**inputs).logits.argmax(-1).item()
print(model.config.id2label[pred_id])

To get a single file like the merged repo:

merged = model.merge_and_unload()
merged.save_pretrained("./bert-emotion-merged")
tokenizer.save_pretrained("./bert-emotion-merged")

That merged folder is what the inference repo already contains.

Training recap

Same run as the merged model:

• Dataset split: 11,200 / 1,600 / 3,200
• Trainable: 2,683,398 params (2.39% of 112,170,252)
• 4 epochs, 1,400 steps, CUDA, ~4.7 min
• Published LR: 2e-4 (2e-5 on the identical adapter underfit: 72.5% val vs 94.1%)
• Test: acc 0.932 / F1 0.933

Learning rate does not change the trainable count. It only changes how far those 2.68M weights move.

Why two repos

Do not create a third “QLoRA” repo for BERT-base. QLoRA is for models that do not fit in VRAM. This 110M encoder already trains in minutes.

License

MIT. Base BERT is Apache 2.0.