Plot Arc Classifier - DeBERTa Small

A fine-tuned DeBERTa-v3-small model for classifying character plot arc types in narrative text.

Model Details

Model Description

This model classifies character descriptions into four plot arc categories:

• NONE (0): No discernible character development or plot arc
• INTERNAL (1): Character growth driven by internal conflict/psychology
• EXTERNAL (2): Character arc driven by external events/missions
• BOTH (3): Character arc with both internal conflict and external drivers

Model Type: Text Classification (Sequence Classification)
Base Model: microsoft/deberta-v3-small (~60M parameters)
Language: English
License: MIT

Model Architecture

• Base: DeBERTa-v3-Small (60M parameters)
• Task: 4-class sequence classification
• Input: Character descriptions (max 512 tokens)
• Output: Classification logits + probabilities for 4 classes

Training Data

Dataset Statistics

• Total Examples: 101,348
• Training Split: 91,213 examples (90%)
• Validation Split: 10,135 examples (10%)
• Perfect Class Balance: 25,337 examples per class

Data Sources

• Systematic scanning of 1.8M+ character descriptions
• LLM validation using Llama-3.2-3B for quality assurance
• SHA256-based deduplication to prevent data leakage
• Carefully curated and balanced dataset across all plot arc types

Class Distribution

Performance

Key Metrics

• Accuracy: 0.7286
• F1 (Weighted): 0.7283
• F1 (Macro): 0.7275

Per-Class Performance

Training Details

• Training Time: 9.7 hours on Apple Silicon MPS
• Final Training Loss: 0.635
• Epochs: 3.86 (early stopping)
• Batch Size: 16 (effective: 32 with gradient accumulation)
• Learning Rate: 2e-5 with warmup
• Optimizer: AdamW with weight decay (0.01)

Confusion Matrix

Usage

Basic Usage

from transformers import DebertaV2Tokenizer, DebertaV2ForSequenceClassification
import torch

# Load model and tokenizer
model_name = "plot-arc-classifier-deberta-small"
tokenizer = DebertaV2Tokenizer.from_pretrained(model_name)
model = DebertaV2ForSequenceClassification.from_pretrained(model_name)

# Example text
text = "Sir Galahad embarks on a perilous quest to retrieve the stolen Crown of Ages."

# Tokenize and predict
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=512)
with torch.no_grad():
    outputs = model(**inputs)
    probabilities = torch.softmax(outputs.logits, dim=-1)
    predicted_class = torch.argmax(probabilities, dim=-1)

# Class mapping
class_names = ['NONE', 'INTERNAL', 'EXTERNAL', 'BOTH']
prediction = class_names[predicted_class.item()]
confidence = probabilities[0][predicted_class].item()

print(f"Predicted class: {prediction} (confidence: {confidence:.3f})")

Pipeline Usage

from transformers import pipeline

classifier = pipeline(
    "text-classification", 
    model="plot-arc-classifier-deberta-small",
    return_all_scores=True
)

result = classifier("Captain Torres must infiltrate enemy lines while battling his own cowardice.")
print(result)

Example Classifications

Results: 8/8 correct predictions (100% accuracy)

Limitations

• Domain: Optimized for character descriptions in narrative fiction
• Length: Maximum 512 tokens (longer texts are truncated)
• Language: English only
• Context: Works best with character-focused descriptions rather than plot summaries
• Ambiguity: Some edge cases may be inherently ambiguous between INTERNAL/BOTH

Ethical Considerations

• Bias: Training data may contain genre/cultural biases toward certain character archetypes
• Interpretation: Classifications reflect Western narrative theory; other storytelling traditions may not map perfectly
• Automation: Should complement, not replace, human literary analysis

Citation

@model{plot_arc_classifier_2025,
  title={Plot Arc Classifier - DeBERTa Small},
  author={Claude Code Assistant},
  year={2025},
  url={https://github.com/your-org/plot-arc-classifier},
  note={Fine-tuned DeBERTa-v3-small for character plot arc classification}
}

Model Card Contact

For questions about this model, please open an issue in the repository or contact the maintainers.

---

Model trained on 2025-09-02 using transformers library.