FOF-80M is a specialized, 80M-parameter text embedding model designed to disentangle factual content from stylistic variation. It is fine-tuned from codefuse-ai/F2LLM-v2-80M to act as a soft factual fingerprint rather than a traditional semantic similarity model.
Standard embedding models often conflate "what is said" with "how it is said", scoring factually altered near-copies higher than stylistically different fact-preserving rewrites. FOF-80M was contrastively trained to invert this behavior: it aggressively collapses stylistic variation (formality, verbosity, rhetorical framing) into a high-similarity space, while forcefully pushing texts with subtle factual alterations (e.g., single entity swaps) apart.
Paper: Focus On The Facts: Stylistically Invariant and Factually Sensitive Text Embeddings https://openreview.net/forum?id=ZnfAvLJRnn
Dataset: MaxDevv/FOF-Bench
Note on Text Normalization: FOF-80M was trained on strictly normalized text to prevent shortcut learning via formatting artifacts (like PDF newlines). Apply the provided clean function to your text before encoding to achieve the documented performance.
from sentence_transformers import SentenceTransformer
import torch
import unicodedata
import re
# 1. Strict Text Normalization
def clean(text):
if not isinstance(text, str): return ""
text = unicodedata.normalize('NFKD', text)
text = text.replace('”', '"').replace('“', '"').replace('’', "'").replace('‘', "'")
text = text.encode('ascii', 'ignore').decode('ascii')
text = re.sub(r'[\n\r\t]+', ' ', text)
return re.sub(r'\s+', ' ', text).strip()
# 2. Load the Model
device = "cuda" if torch.cuda.is_available() else "cpu"
model = SentenceTransformer("MaxDevv/FOF-80M", trust_remote_code=True, device=device)
# Explicit 16-bit conversion is recommended for memory efficiency
if torch.cuda.is_available():
model.bfloat16() if torch.cuda.is_bf16_supported() else model.half()
# 3. Example Texts
anchor = "We present ab initio calculations of the electronic energy loss of charged particles moving outside a magnesium surface."
altered_copy = "We present ab initio calculations of the electronic energy loss of charged particles moving outside a beryllium surface."
rewrite = "In this highly detailed scientific work, we put forward a set of comprehensive ab initio calculations pertaining to the electronic energy loss experienced by charged particles as they travel in close proximity to, but outside of, a magnesium surface."
# Clean and Encode
texts = [clean(t) for t in [anchor, altered_copy, rewrite]]
embeddings = model.encode(texts, normalize_embeddings=True)
# 4. Compare
sim_altered = (embeddings[0] @ embeddings[1].T).item()
sim_rewrite = (embeddings[0] @ embeddings[2].T).item()
print(f"Similarity to Fact-Altered Copy: {sim_altered:.4f}")
print(f"Similarity to Fact-Preserving Rewrite: {sim_rewrite:.4f}")
FOF-80M dramatically outperforms its base model at recognizing factual parity across stylistic gaps, but it trades off performance on tasks requiring tolerance for partial information.
| Benchmark | Metric | Base Model (F2LLM-v2-80M) |
FOF-80M |
|---|---|---|---|
| FOF-Bench (Factual Parity) | Triplet Accuracy | 0.42% | 29.37% |
| Mean Separation (Pos - Neg) | -0.1650 | +0.0329 | |
| DiSC (Stylistic Invariance) | Mean Cross-Style Sim | 0.7905 | 0.9559 |
| SNLI (NLI Probing) | 3-Way Accuracy (LR) | 71.07% | 52.91% |
| SummEval (Summarization) | Consistency Correlation (ρ) | 0.2061 | 0.0198 |
FOF-80M has a highly distinct, "hash-like" capability profile. It is not a general-purpose semantic embedding model.
If you use this model, please cite the accompanying paper:
@inproceedings{
adeola2026focus,
title={Focus On Facts: Stylistically Invariant and Factually Sensitive Text Embeddings},
author={Maximus Adeola},
booktitle={NeurIPS 2026 Workshop on Linguistic Principles for Foundation Models},
year={2026},
url={https://openreview.net/forum?id=ZnfAvLJRnn}
}
checkpoint-10000/model.safetensors 320 MBcheckpoint-11000/model.safetensors 320 MBcheckpoint-12000/model.safetensors 320 MBcheckpoint-4000/model.safetensors 320 MBcheckpoint-5000/model.safetensors 320 MBcheckpoint-6000/model.safetensors 320 MBcheckpoint-7000/model.safetensors 320 MBcheckpoint-8000/model.safetensors 320 MBcheckpoint-9000/model.safetensors 320 MBmodel.safetensors 320 MBcheckpoint-10000/training_args.bin 6.1 kBcheckpoint-11000/training_args.bin 6.1 kBcheckpoint-12000/training_args.bin 6.1 kBcheckpoint-4000/training_args.bin 6.1 kBcheckpoint-5000/training_args.bin 6.1 kBcheckpoint-6000/training_args.bin 6.1 kBcheckpoint-7000/training_args.bin 6.1 kBcheckpoint-8000/training_args.bin 6.1 kBcheckpoint-9000/training_args.bin 6.1 kBtraining_args.bin 6.1 kBHave the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo MaxDevv/FOF-80M ./model-foldercodefuse-ai/F2LLM-v2-80M