multilingual DistilBERT finetuned on PLOTS
distilbert/distilbert-base-multilingual-cased finetuned on PLOTS, pairs of independently written Wikipedia plot summaries of the same film (English plot paired with its plot in another language), to match stories rather than words or names.
Training
197,694 training pairs; symmetric in-batch contrastive loss (InfoNCE in both directions) with gradient caching, batch 128, 1 epoch, learning rate 5e-05, maximum input length 512 tokens, seed 13. Films of SemEval-2026 Task 4 and TREC ToT 2023 (train and dev targets) are excluded from training.
Results
NS: accuracy on the 355 leak-free SemEval-2026 Task 4 track-B test triples. PLOTS-IR: nDCG@10 on the test queries (M: entity-masked). ToT: TREC ToT 2023 dev queries.
| NS acc. | Full | Full M | First | Last | Last M | ToT nDCG@10 | ToT R@100 | |
|---|---|---|---|---|---|---|---|---|
| this model | 0.575 | 0.955 | 0.549 | 0.717 | 0.281 | 0.024 | 0.036 | 0.147 |
| multilingual DistilBERT, zero-shot | 0.572 | 0.182 | 0.091 | 0.034 | 0.016 | 0.008 | 0.000 | 0.013 |
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("Jgmorenof/plots-distilbert-base-multilingual-cased", trust_remote_code=True)
queries = ["<plot summary or description of a film, in any language>"]
documents = ["<English plot summary>"]
q = model.encode(queries, prompt="")
D = model.encode(documents, prompt="")
scores = model.similarity(q, D)
- Reference model only: the base checkpoint has no retrieval pretraining, and the paper shows that it benefits far less from PLOTS than retrieval-pretrained encoders.
License
The weights keep the license of the base model (apache-2.0). The training data (PLOTS) is CC BY-SA 4.0.
Citation
@misc{moreno2026plots,
title = {{PLOTS} -- Same Story, Different Words: Paired Plot Outlines for Narrative Similarity and Retrieval},
author = {Moreno, Jose G. and Hamdi, Ahmed and Boros, Emanuela and Jatowt, Adam and Bordea, Georgeta and Gonz{\'a}lez-Gallardo, Carlos-Emiliano and Doucet, Antoine},
year = {2026},
howpublished = {\url{https://huggingface.co/datasets/Jgmorenof/PLOTS}}
}