Qwen3-Embedding-8B (LoRA, type-masked training) finetuned on PLOTS
Qwen/Qwen3-Embedding-8B finetuned on PLOTS, pairs of independently written Wikipedia plot summaries of the same film (English plot paired with its plot in another language), to match stories rather than words or names.
Training
20,000 training pairs (a random subset), type-masked plots in fr, es, pt, de, it; symmetric in-batch contrastive loss (InfoNCE in both directions) with gradient caching, batch 128, 1 epoch, learning rate 0.0001, maximum input length 512 tokens, LoRA rank 16, seed 13. Films of SemEval-2026 Task 4 and TREC ToT 2023 (train and dev targets) are excluded from training.
Results
NS: accuracy on the 355 leak-free SemEval-2026 Task 4 track-B test triples. PLOTS-IR: nDCG@10 on the test queries (M: entity-masked). ToT: TREC ToT 2023 dev queries.
| NS acc. | Full | Full M | First | Last | Last M | ToT nDCG@10 | ToT R@100 | |
|---|---|---|---|---|---|---|---|---|
| this model | 0.693 | 0.991 | 0.972 | 0.725 | 0.332 | 0.184 | 0.370 | 0.700 |
| Qwen3-Embedding-8B, zero-shot | 0.642 | 0.991 | 0.912 | 0.848 | 0.512 | 0.183 | 0.241 | 0.580 |
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("Qwen/Qwen3-Embedding-8B", trust_remote_code=True)
model.load_adapter("Jgmorenof/plots-qwen3-embedding-8b-lora-masked")
queries = ["<plot summary or description of a film, in any language>"]
documents = ["<English plot summary>"]
q = model.encode(queries, prompt="Instruct: Given a story summary, retrieve the plot summary of the same film\nQuery:")
D = model.encode(documents, prompt="")
scores = model.similarity(q, D)
- For tip-of-the-tongue requests we used the query prompt "Instruct: Given a description of a movie someone half-remembers, retrieve the movie's Wikipedia page\nQuery:"; for symmetric story-to-story similarity (SemEval) "Instruct: Retrieve semantically similar text\nQuery:" on both sides.
- This repository holds a LoRA adapter (rank 16); it needs the base model and
peft. - Trained on type-masked plots (entities replaced by PERSON, LOCATION, ORG with spaCy, as in the
maskedconfig of the dataset). It is the strongest model of the paper on TREC ToT 2023 but weaker on short unmasked queries that can only be matched through names.
License
The weights keep the license of the base model (apache-2.0). The training data (PLOTS) is CC BY-SA 4.0.
Citation
@misc{moreno2026plots,
title = {{PLOTS} -- Same Story, Different Words: Paired Plot Outlines for Narrative Similarity and Retrieval},
author = {Moreno, Jose G. and Hamdi, Ahmed and Boros, Emanuela and Jatowt, Adam and Bordea, Georgeta and Gonz{\'a}lez-Gallardo, Carlos-Emiliano and Doucet, Antoine},
year = {2026},
howpublished = {\url{https://huggingface.co/datasets/Jgmorenof/PLOTS}}
}