Voicemail Detection Model (3-Utterance)

Binary classification model to detect voicemail vs human on phone calls.

Performance

Validation Set

• Accuracy: 0.9703
• Precision: 0.9005
• Recall: 0.9794
• F1: 0.9383

Test Set

• Accuracy: 0.8353
• Precision: 0.6678
• Recall: 0.9895
• F1: 0.7975

Details

Base: prajjwal1/bert-tiny Threshold: 0.1153 Training: 2025-10-04

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "Adya662/bert-tiny-amd"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
model.eval()

text = "Hi you've reached voicemail"
encoding = tokenizer(
    text,
    return_tensors='pt',
    max_length=128,
    padding='max_length',
    truncation=True
)

with torch.no_grad():
    outputs = model(**encoding)
    # Assuming label 1 = voicemail (update if different)
    probs = torch.softmax(outputs.logits, dim=-1)
    probability = probs[0, 1].item()

optimal_threshold = 0.1153
prediction = "voicemail" if probability >= optimal_threshold else "human"
print({"probability": probability, "prediction": prediction})