ColBERT ModernBERT-base trained on MS MARCO triplets with token pooling regularization

This is a Multi-Vector Encoder model finetuned from answerdotai/ModernBERT-base on the msmarco-bm25 dataset using the sentence-transformers library. It maps inputs to sequences of 128-dimensional token-level vectors and scores them with late interaction (MaxSim), useful for semantic search with late interaction.

Model Details

Model Description

Model Type: Multi-Vector Encoder
Base model: answerdotai/ModernBERT-base
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 128 dimensions
Similarity Function: MaxSim
Supported Modality: Text
Training Dataset:
- msmarco-bm25
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Documentation: Multi-Vector Encoder Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Multi-Vector Encoders on Hugging Face

Full Model Architecture

MultiVectorEncoder(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'query_expansion': {'strategy': 'min', 'attend': False, 'token': None, 'length': 32}, 'architecture': 'ModernBertModel'})
  (1): Dense({'in_features': 768, 'out_features': 128, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'module_input_name': 'token_embeddings', 'module_output_name': 'token_embeddings'})
  (2): MultiVectorMask({'skiplist_words': [], 'skiplist_tasks': ['document'], 'keep_only_token_ids': None})
  (3): Normalize({'module_input_name': 'token_embeddings', 'module_output_name': 'token_embeddings'})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import MultiVectorEncoder

# Download from the 🤗 Hub
model = MultiVectorEncoder("tomaarsen/multivector-ModernBERT-base-msmarco-cached-contrastive-token-pooling")
# Run inference: each input becomes a sequence of per-token vectors (variable length).
queries = [
    'what county is ireton iowa in',
]
documents = [
    'For the surname, see Ireton (surname). Ireton is a city in Sioux County, Iowa, United States. The population was 609 at the 2010 census.',
    'For the surname, see Ireton (surname). Ireton is a city in Sioux County, Iowa, United States. The population was 609 at the 2010 census.',
    'parsimonious meaning, definition, what is parsimonious: not willing to spend money or give something: . Learn more.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings[0].shape, document_embeddings[0].shape)
# (32, 128) (39, 128)

# Get the MaxSim similarity scores
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[21.0361, 21.0361,  5.3709]])

Evaluation

Metrics

Multi Vector Information Retrieval

Datasets: NanoMSMARCO, NanoNQ, NanoFiQA2018, NanoClimateFEVER, NanoDBPedia, NanoFEVER, NanoFiQA2018, NanoHotpotQA, NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoQuoraRetrieval, NanoSCIDOCS, NanoArguAna, NanoSciFact, NanoTouche2020, NanoClimateFEVER_pool_factor_1.5, NanoDBPedia_pool_factor_1.5, NanoFEVER_pool_factor_1.5, NanoFiQA2018_pool_factor_1.5, NanoHotpotQA_pool_factor_1.5, NanoMSMARCO_pool_factor_1.5, NanoNFCorpus_pool_factor_1.5, NanoNQ_pool_factor_1.5, NanoQuoraRetrieval_pool_factor_1.5, NanoSCIDOCS_pool_factor_1.5, NanoArguAna_pool_factor_1.5, NanoSciFact_pool_factor_1.5, NanoTouche2020_pool_factor_1.5, NanoClimateFEVER_pool_factor_2, NanoDBPedia_pool_factor_2, NanoFEVER_pool_factor_2, NanoFiQA2018_pool_factor_2, NanoHotpotQA_pool_factor_2, NanoMSMARCO_pool_factor_2, NanoNFCorpus_pool_factor_2, NanoNQ_pool_factor_2, NanoQuoraRetrieval_pool_factor_2, NanoSCIDOCS_pool_factor_2, NanoArguAna_pool_factor_2, NanoSciFact_pool_factor_2, NanoTouche2020_pool_factor_2, NanoClimateFEVER_pool_factor_3, NanoDBPedia_pool_factor_3, NanoFEVER_pool_factor_3, NanoFiQA2018_pool_factor_3, NanoHotpotQA_pool_factor_3, NanoMSMARCO_pool_factor_3, NanoNFCorpus_pool_factor_3, NanoNQ_pool_factor_3, NanoQuoraRetrieval_pool_factor_3, NanoSCIDOCS_pool_factor_3, NanoArguAna_pool_factor_3, NanoSciFact_pool_factor_3, NanoTouche2020_pool_factor_3, NanoClimateFEVER_pool_factor_4, NanoDBPedia_pool_factor_4, NanoFEVER_pool_factor_4, NanoFiQA2018_pool_factor_4, NanoHotpotQA_pool_factor_4, NanoMSMARCO_pool_factor_4, NanoNFCorpus_pool_factor_4, NanoNQ_pool_factor_4, NanoQuoraRetrieval_pool_factor_4, NanoSCIDOCS_pool_factor_4, NanoArguAna_pool_factor_4, NanoSciFact_pool_factor_4 and NanoTouche2020_pool_factor_4
Evaluated with MultiVectorInformationRetrievalEvaluator

Metric	NanoMSMARCO	NanoNQ	NanoFiQA2018	NanoClimateFEVER	NanoDBPedia	NanoFEVER	NanoHotpotQA	NanoNFCorpus	NanoQuoraRetrieval	NanoSCIDOCS	NanoArguAna	NanoSciFact	NanoTouche2020	NanoClimateFEVER_pool_factor_1.5	NanoDBPedia_pool_factor_1.5	NanoFEVER_pool_factor_1.5	NanoFiQA2018_pool_factor_1.5	NanoHotpotQA_pool_factor_1.5	NanoMSMARCO_pool_factor_1.5	NanoNFCorpus_pool_factor_1.5	NanoNQ_pool_factor_1.5	NanoQuoraRetrieval_pool_factor_1.5	NanoSCIDOCS_pool_factor_1.5	NanoArguAna_pool_factor_1.5	NanoSciFact_pool_factor_1.5	NanoTouche2020_pool_factor_1.5	NanoClimateFEVER_pool_factor_2	NanoDBPedia_pool_factor_2	NanoFEVER_pool_factor_2	NanoFiQA2018_pool_factor_2	NanoHotpotQA_pool_factor_2	NanoMSMARCO_pool_factor_2	NanoNFCorpus_pool_factor_2	NanoNQ_pool_factor_2	NanoQuoraRetrieval_pool_factor_2	NanoSCIDOCS_pool_factor_2	NanoArguAna_pool_factor_2	NanoSciFact_pool_factor_2	NanoTouche2020_pool_factor_2	NanoClimateFEVER_pool_factor_3	NanoDBPedia_pool_factor_3	NanoFEVER_pool_factor_3	NanoFiQA2018_pool_factor_3	NanoHotpotQA_pool_factor_3	NanoMSMARCO_pool_factor_3	NanoNFCorpus_pool_factor_3	NanoNQ_pool_factor_3	NanoQuoraRetrieval_pool_factor_3	NanoSCIDOCS_pool_factor_3	NanoArguAna_pool_factor_3	NanoSciFact_pool_factor_3	NanoTouche2020_pool_factor_3	NanoClimateFEVER_pool_factor_4	NanoDBPedia_pool_factor_4	NanoFEVER_pool_factor_4	NanoFiQA2018_pool_factor_4	NanoHotpotQA_pool_factor_4	NanoMSMARCO_pool_factor_4	NanoNFCorpus_pool_factor_4	NanoNQ_pool_factor_4	NanoQuoraRetrieval_pool_factor_4	NanoSCIDOCS_pool_factor_4	NanoArguAna_pool_factor_4	NanoSciFact_pool_factor_4	NanoTouche2020_pool_factor_4
maxsim_accuracy@1	0.32	0.38	0.34	0.18	0.52	0.64	0.7	0.4	0.82	0.34	0.1	0.48	0.449	0.16	0.54	0.66	0.3	0.68	0.3	0.4	0.32	0.8	0.36	0.08	0.46	0.3878	0.2	0.5	0.64	0.36	0.68	0.32	0.4	0.34	0.8	0.34	0.08	0.46	0.449	0.18	0.5	0.62	0.34	0.66	0.34	0.4	0.28	0.74	0.34	0.08	0.52	0.4286	0.18	0.5	0.64	0.34	0.64	0.38	0.38	0.26	0.56	0.36	0.08	0.48	0.4694
maxsim_accuracy@3	0.5	0.58	0.5	0.38	0.76	0.84	0.86	0.48	0.98	0.58	0.4	0.68	0.7551	0.36	0.66	0.84	0.52	0.86	0.52	0.46	0.56	0.98	0.62	0.4	0.72	0.7551	0.4	0.68	0.86	0.52	0.84	0.52	0.46	0.62	0.94	0.6	0.4	0.66	0.7551	0.42	0.72	0.88	0.5	0.84	0.5	0.44	0.56	0.86	0.66	0.42	0.7	0.7143	0.38	0.72	0.9	0.46	0.8	0.5	0.46	0.58	0.8	0.54	0.36	0.7	0.7959
maxsim_accuracy@5	0.6	0.68	0.54	0.52	0.82	0.96	0.9	0.5	1.0	0.72	0.54	0.82	0.8367	0.54	0.76	0.92	0.54	0.9	0.58	0.52	0.66	1.0	0.72	0.52	0.8	0.8367	0.56	0.82	0.92	0.54	0.9	0.56	0.5	0.68	1.0	0.74	0.52	0.76	0.8571	0.56	0.8	0.96	0.52	0.9	0.54	0.52	0.68	0.92	0.7	0.52	0.76	0.8163	0.54	0.8	0.96	0.56	0.88	0.52	0.54	0.66	0.9	0.66	0.5	0.78	0.8367
maxsim_accuracy@10	0.8	0.82	0.6	0.68	0.9	0.98	0.96	0.62	1.0	0.8	0.62	0.84	0.9388	0.66	0.9	0.98	0.58	0.96	0.76	0.62	0.78	1.0	0.78	0.6	0.84	0.9592	0.66	0.96	0.98	0.58	0.94	0.76	0.6	0.8	1.0	0.78	0.62	0.84	0.9592	0.62	0.96	0.98	0.56	0.94	0.78	0.64	0.82	0.96	0.82	0.64	0.86	0.898	0.68	0.96	0.98	0.58	0.9	0.76	0.6	0.78	0.96	0.82	0.64	0.84	0.9184
maxsim_precision@1	0.32	0.38	0.34	0.18	0.52	0.64	0.7	0.4	0.82	0.34	0.1	0.48	0.449	0.16	0.54	0.66	0.3	0.68	0.3	0.4	0.32	0.8	0.36	0.08	0.46	0.3878	0.2	0.5	0.64	0.36	0.68	0.32	0.4	0.34	0.8	0.34	0.08	0.46	0.449	0.18	0.5	0.62	0.34	0.66	0.34	0.4	0.28	0.74	0.34	0.08	0.52	0.4286	0.18	0.5	0.64	0.34	0.64	0.38	0.38	0.26	0.56	0.36	0.08	0.48	0.4694
maxsim_precision@3	0.1667	0.1933	0.2267	0.1333	0.4933	0.2933	0.4067	0.3467	0.4	0.2733	0.1333	0.2333	0.4898	0.1267	0.44	0.2933	0.2267	0.4133	0.1733	0.32	0.1867	0.4	0.2867	0.1333	0.2467	0.4966	0.14	0.4333	0.3	0.2333	0.4067	0.1733	0.34	0.2067	0.38	0.2933	0.1333	0.2333	0.4898	0.1533	0.4533	0.3067	0.2067	0.3667	0.1667	0.3067	0.1867	0.3533	0.3	0.14	0.2467	0.4626	0.1467	0.4467	0.3133	0.2	0.36	0.1667	0.3133	0.1933	0.2867	0.2667	0.12	0.2467	0.5102
maxsim_precision@5	0.12	0.136	0.164	0.12	0.444	0.2	0.272	0.3	0.252	0.252	0.108	0.176	0.4408	0.124	0.42	0.192	0.164	0.272	0.116	0.3	0.136	0.252	0.252	0.104	0.168	0.4367	0.128	0.44	0.192	0.164	0.272	0.112	0.296	0.136	0.252	0.252	0.104	0.164	0.4612	0.124	0.444	0.2	0.148	0.256	0.108	0.288	0.136	0.232	0.24	0.104	0.164	0.4367	0.124	0.424	0.2	0.152	0.252	0.104	0.288	0.132	0.204	0.232	0.1	0.168	0.449
maxsim_precision@10	0.08	0.086	0.106	0.08	0.426	0.102	0.154	0.242	0.134	0.168	0.062	0.092	0.4061	0.078	0.426	0.102	0.098	0.152	0.076	0.242	0.082	0.13	0.162	0.06	0.094	0.3816	0.078	0.432	0.102	0.1	0.148	0.076	0.232	0.082	0.132	0.16	0.062	0.092	0.3918	0.076	0.428	0.102	0.09	0.148	0.078	0.234	0.084	0.122	0.16	0.064	0.094	0.3837	0.08	0.39	0.102	0.092	0.14	0.076	0.23	0.08	0.126	0.156	0.064	0.092	0.3816
maxsim_recall@1	0.32	0.36	0.1897	0.0817	0.0705	0.5967	0.35	0.0428	0.7207	0.0727	0.1	0.455	0.0357	0.0633	0.0712	0.6267	0.1697	0.34	0.3	0.0428	0.3	0.7007	0.0767	0.08	0.435	0.0304	0.1017	0.0672	0.6067	0.1964	0.34	0.32	0.0428	0.32	0.7107	0.0717	0.08	0.435	0.0348	0.0817	0.0706	0.5867	0.1997	0.33	0.34	0.0428	0.26	0.6373	0.069	0.08	0.495	0.0318	0.0817	0.0701	0.6067	0.1897	0.32	0.38	0.0417	0.25	0.4867	0.073	0.08	0.455	0.0369
maxsim_recall@3	0.5	0.55	0.3178	0.1817	0.1395	0.7967	0.61	0.0771	0.9387	0.1687	0.4	0.66	0.1146	0.1723	0.1264	0.7967	0.325	0.62	0.52	0.0741	0.53	0.9387	0.1767	0.4	0.69	0.1141	0.1917	0.1247	0.8167	0.3278	0.61	0.52	0.0763	0.58	0.8887	0.1807	0.4	0.645	0.1094	0.199	0.1283	0.8367	0.3196	0.55	0.5	0.0735	0.53	0.8153	0.1867	0.42	0.665	0.1068	0.184	0.1175	0.8567	0.318	0.54	0.5	0.0741	0.54	0.7107	0.1667	0.36	0.675	0.1183
maxsim_recall@5	0.6	0.63	0.3723	0.2517	0.1803	0.9167	0.68	0.0953	0.9693	0.2587	0.54	0.805	0.1669	0.254	0.1707	0.8767	0.3723	0.68	0.58	0.0982	0.62	0.9693	0.2587	0.52	0.775	0.1668	0.2573	0.1802	0.8767	0.3738	0.68	0.56	0.0924	0.62	0.9693	0.2597	0.52	0.735	0.1757	0.2557	0.1788	0.9167	0.353	0.64	0.54	0.0931	0.62	0.8853	0.2477	0.52	0.73	0.17	0.254	0.1708	0.9167	0.373	0.63	0.52	0.0932	0.6	0.8147	0.2397	0.5	0.75	0.1743
maxsim_recall@10	0.8	0.77	0.4377	0.3107	0.2799	0.9367	0.77	0.1255	0.9893	0.3437	0.62	0.82	0.2923	0.3057	0.2829	0.9367	0.4133	0.76	0.76	0.1233	0.73	0.9793	0.3317	0.6	0.83	0.279	0.3073	0.32	0.9367	0.4268	0.74	0.76	0.1207	0.74	0.986	0.3277	0.62	0.82	0.2798	0.2973	0.3079	0.9367	0.3986	0.74	0.78	0.124	0.76	0.932	0.3277	0.64	0.84	0.2817	0.319	0.3021	0.9367	0.4066	0.7	0.76	0.1197	0.72	0.9393	0.3197	0.64	0.82	0.2782
maxsim_ndcg@10	0.5373	0.5586	0.3746	0.2397	0.5019	0.7899	0.692	0.3052	0.9145	0.3207	0.3649	0.6479	0.4434	0.2291	0.4925	0.7948	0.3479	0.6762	0.515	0.3013	0.5258	0.9012	0.32	0.3529	0.6453	0.4223	0.2487	0.5029	0.7905	0.3715	0.659	0.5252	0.295	0.5409	0.8932	0.3126	0.3543	0.6404	0.4361	0.2387	0.5019	0.7908	0.3523	0.6443	0.5269	0.2943	0.5241	0.8293	0.3129	0.3644	0.6677	0.4248	0.2391	0.477	0.8043	0.3508	0.6216	0.5414	0.2881	0.4905	0.7381	0.3056	0.3464	0.6509	0.4318
maxsim_mrr@10	0.4568	0.5077	0.4268	0.321	0.6559	0.7647	0.7952	0.4534	0.8983	0.4794	0.2827	0.601	0.6236	0.3049	0.63	0.7657	0.395	0.781	0.4389	0.4517	0.4746	0.8883	0.4998	0.2724	0.5915	0.5974	0.3389	0.621	0.763	0.4367	0.7653	0.453	0.4457	0.4981	0.873	0.4837	0.2686	0.5899	0.6276	0.3219	0.6318	0.7626	0.416	0.7565	0.4514	0.452	0.4672	0.8174	0.4941	0.2761	0.6221	0.5966	0.3157	0.6372	0.7792	0.4163	0.738	0.4764	0.4367	0.4304	0.698	0.4824	0.2541	0.6041	0.6281
maxsim_map@100	0.4661	0.4912	0.3246	0.1797	0.3888	0.7344	0.6088	0.1401	0.8814	0.2361	0.3003	0.5933	0.3787	0.1696	0.3818	0.7442	0.3026	0.5883	0.4517	0.1375	0.4625	0.8679	0.2392	0.2912	0.5853	0.3697	0.1929	0.3818	0.7372	0.3248	0.5736	0.4645	0.1377	0.4773	0.8551	0.233	0.2856	0.5838	0.3745	0.186	0.3796	0.7358	0.3159	0.5487	0.4628	0.1383	0.4476	0.7923	0.2297	0.2917	0.6126	0.3744	0.178	0.357	0.7545	0.3071	0.5364	0.4868	0.1366	0.4204	0.6613	0.2259	0.2698	0.5982	0.3813

Multi Vector Nano BEIR

Dataset: NanoBEIR_mean

Evaluated with MultiVectorNanoBEIREvaluator with these parameters:

{
    "dataset_names": [
        "msmarco",
        "nq",
        "fiqa2018"
    ],
    "dataset_id": "sentence-transformers/NanoBEIR-en"
}

Metric	Value
maxsim_accuracy@1	0.3467
maxsim_accuracy@3	0.5267
maxsim_accuracy@5	0.6067
maxsim_accuracy@10	0.74
maxsim_precision@1	0.3467
maxsim_precision@3	0.1956
maxsim_precision@5	0.14
maxsim_precision@10	0.0907
maxsim_recall@1	0.2899
maxsim_recall@3	0.4559
maxsim_recall@5	0.5341
maxsim_recall@10	0.6692
maxsim_ndcg@10	0.4901
maxsim_mrr@10	0.4638
maxsim_map@100	0.4273

Multi Vector Nano BEIR

Datasets: NanoBEIR_mean, NanoBEIR_mean_pool_factor_1.5, NanoBEIR_mean_pool_factor_2, NanoBEIR_mean_pool_factor_3 and NanoBEIR_mean_pool_factor_4

Evaluated with MultiVectorNanoBEIREvaluator with these parameters:

{
    "dataset_names": [
        "climatefever",
        "dbpedia",
        "fever",
        "fiqa2018",
        "hotpotqa",
        "msmarco",
        "nfcorpus",
        "nq",
        "quoraretrieval",
        "scidocs",
        "arguana",
        "scifact",
        "touche2020"
    ],
    "dataset_id": "sentence-transformers/NanoBEIR-en"
}

Metric	NanoBEIR_mean	NanoBEIR_mean_pool_factor_1.5	NanoBEIR_mean_pool_factor_2	NanoBEIR_mean_pool_factor_3	NanoBEIR_mean_pool_factor_4
maxsim_accuracy@1	0.4361	0.4191	0.4284	0.4176	0.4053
maxsim_accuracy@3	0.6381	0.635	0.635	0.6319	0.6151
maxsim_accuracy@5	0.7259	0.7151	0.7198	0.7074	0.7028
maxsim_accuracy@10	0.8122	0.8015	0.8061	0.806	0.8014
maxsim_precision@1	0.4361	0.4191	0.4284	0.4176	0.4053
maxsim_precision@3	0.2915	0.2879	0.2895	0.2807	0.2746
maxsim_precision@5	0.2296	0.2259	0.2287	0.2216	0.2176
maxsim_precision@10	0.1645	0.1603	0.1606	0.1587	0.1546
maxsim_recall@1	0.2612	0.249	0.2559	0.248	0.2363
maxsim_recall@3	0.4196	0.4218	0.4208	0.4101	0.397
maxsim_recall@5	0.4974	0.4878	0.4846	0.4731	0.4643
maxsim_recall@10	0.5766	0.564	0.5681	0.5666	0.5586
maxsim_ndcg@10	0.5147	0.5019	0.5054	0.4979	0.4835
maxsim_mrr@10	0.559	0.5455	0.5511	0.5435	0.5305
maxsim_map@100	0.4403	0.4301	0.4324	0.4243	0.4087

Training Details

Training Dataset

msmarco-bm25

Dataset: msmarco-bm25 at ce8a493
Size: 99,000 training samples
Columns: query, positive, and negative

Approximate statistics based on the first 100 samples:

	query	positive	negative
type	string	string	string
modality	text	text	text
details	min: 5 tokens mean: 9.4 tokens max: 18 tokens	min: 28 tokens mean: 83.3 tokens max: 197 tokens	min: 24 tokens mean: 76.0 tokens max: 226 tokens

Samples:

query	positive	negative
`what is an agate made of`	`Agate is the name given to a group of silicate minerals that are made up primarily of chalcedony. Chalcedony is a member of the quartz family of minerals.`	`What is an agate, though, and what are the properties and feng shui meaning of agate? Let's find out. WHAT IS THE MEANING OF AGATE? As a form of chalcedony (type of quartz) agate exhibits a variety of colours , shapes, as well as an often present gentle iridescence.`
`what is the socratic method?`	`The Socratic Learning Method (SLM) is a constructivist learning approach consisting of four key. steps: eliciting relevant preconceptions, clarifying preconceptions, testing ones own. hypotheses or encountered propositions, and deciding whether to accept the hypotheses or. propositions.`	`The Socratic Learning Method and the Inquiry-Based Learning Method. Since the Socratic dialogues are among the earliest documented instances of learning through. inquiry, it is reasonable to argue that what is now known as inquiry-based learning can trace its. origin to the Socratic Learning Method.`
`what ditto means`	`â¢ DITTO (noun). The noun DITTO has 1 sense: 1. a mark used to indicate the word above it should be repeated. Familiarity information: DITTO used as a noun is very rare. â¢ DITTO (verb). The verb DITTO has 1 sense: 1. repeat an action or statement. Familiarity information: DITTO used as a verb is very rare.`	`Valerie Hill 0 I had texted one of my Auntie's and she sent the word Ditto back to me I was like what do that mean. Now I know!!! Acronym. DITTO in text means the same, or me too, or I agree.`

Loss: MultiVectorTokenPoolingLoss with these parameters:

{
    "loss": "CachedMultiVectorMultipleNegativesRankingLoss",
    "poolings": [
        "HierarchicalTokenPooling(pool_factor=1, max_tokens=None, num_protected_tokens=1, tasks=['document'])",
        "HierarchicalTokenPooling(pool_factor=1.5, max_tokens=None, num_protected_tokens=1, tasks=['document'])",
        "HierarchicalTokenPooling(pool_factor=2, max_tokens=None, num_protected_tokens=1, tasks=['document'])",
        "HierarchicalTokenPooling(pool_factor=2.5, max_tokens=None, num_protected_tokens=1, tasks=['document'])",
        "HierarchicalTokenPooling(pool_factor=3, max_tokens=None, num_protected_tokens=1, tasks=['document'])",
        "HierarchicalTokenPooling(pool_factor=3.5, max_tokens=None, num_protected_tokens=1, tasks=['document'])",
        "HierarchicalTokenPooling(pool_factor=4, max_tokens=None, num_protected_tokens=1, tasks=['document'])"
    ],
    "weights": [
        1,
        1,
        1,
        1,
        1,
        1,
        1
    ],
    "n_poolings_per_step": -1
}

Evaluation Dataset

msmarco-bm25

Dataset: msmarco-bm25 at ce8a493
Size: 1,000 evaluation samples
Columns: query, positive, and negative

Approximate statistics based on the first 100 samples:

	query	positive	negative
type	string	string	string
modality	text	text	text
details	min: 4 tokens mean: 9.18 tokens max: 21 tokens	min: 22 tokens mean: 76.76 tokens max: 210 tokens	min: 30 tokens mean: 84.36 tokens max: 210 tokens

Samples:

query	positive	negative
`expected return is a function of`	`security market line Security market line (SML) is the representation of the capital asset pricing model. It displays the expected rate of return of an individual security as a function of systematic, non-diversifiable risk (its beta). beta Average sensitivity of a security's price to overall securities market prices.`	`A portfolio's expected return is the sum of the weighted average of each asset's expected return. Calculate a portfolio's expected return. To calculate the expected return of a portfolio, you need to know the expected return and weight of each asset in a portfolio.`
`what smoker temperature for barbecue chicken`	`Chickens smoked hot and fast over indirect heat on the grill. But for pulled chicken, I wanted a slightly more intense smokiness to balance the barbecue sauce. I decided to do a side-by-side comparison, cooking one on the grill over indirect heat at around 375Â°F, and one in the smoker at 225Â°F.`	`Cook for an hour and 15 minutes. Check your smoked chicken breasts to make sure the temperature is still holding at about 250 degrees. Flip the chicken breasts and close the lid again. After 30 minutes, check the internal temperature of the chicken with a meat thermometer. You are looking for a temperature of 160 degrees before you can pull them off the smoker. Serve your smoked chicken breasts with a side of barbecue sauce for dipping.`
`in classification of matter what is an element`	`Matter can be in the same phase or in two different phases for this separation to take place. Key Terms. mixture: Something that consists of diverse, non-bonded elements or molecules. element: A chemical substance that is made up of a particular kind of atom and cannot be broken down or transformed by a chemical reaction.`	2. What four elements make up 96% of all living matter? The four elements that make up 96% of all living matter are oxygen, carbon, hydrogen and nitrogen. 3. What is the difference between an essential element and a trace element? An essential element is an element that an organism needs to live a healthy life and reproduce. A trace element is required by an organism in only minute quantities. Section 2 4. Sketch a model of an atom of helium, showing the electrons, protons, neutrons, and atomic nucleus. Neutrons Protons Electrons 5.

Loss: MultiVectorTokenPoolingLoss with these parameters:

{
    "loss": "CachedMultiVectorMultipleNegativesRankingLoss",
    "poolings": [
        "HierarchicalTokenPooling(pool_factor=1, max_tokens=None, num_protected_tokens=1, tasks=['document'])",
        "HierarchicalTokenPooling(pool_factor=1.5, max_tokens=None, num_protected_tokens=1, tasks=['document'])",
        "HierarchicalTokenPooling(pool_factor=2, max_tokens=None, num_protected_tokens=1, tasks=['document'])",
        "HierarchicalTokenPooling(pool_factor=2.5, max_tokens=None, num_protected_tokens=1, tasks=['document'])",
        "HierarchicalTokenPooling(pool_factor=3, max_tokens=None, num_protected_tokens=1, tasks=['document'])",
        "HierarchicalTokenPooling(pool_factor=3.5, max_tokens=None, num_protected_tokens=1, tasks=['document'])",
        "HierarchicalTokenPooling(pool_factor=4, max_tokens=None, num_protected_tokens=1, tasks=['document'])"
    ],
    "weights": [
        1,
        1,
        1,
        1,
        1,
        1,
        1
    ],
    "n_poolings_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 256
num_train_epochs: 1
learning_rate: 3e-05
warmup_steps: 0.05
bf16: True
per_device_eval_batch_size: 32
load_best_model_at_end: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

per_device_train_batch_size: 256
num_train_epochs: 1
max_steps: -1
learning_rate: 3e-05
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_steps: 0.05
optim: adamw_torch_fused
optim_args: None
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
optim_target_modules: None
gradient_accumulation_steps: 1
average_tokens_across_devices: True
max_grad_norm: 1.0
label_smoothing_factor: 0.0
bf16: True
fp16: False
bf16_full_eval: False
fp16_full_eval: False
tf32: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
use_liger_kernel: False
liger_kernel_config: None
use_cache: False
neftune_noise_alpha: None
torch_empty_cache_steps: None
auto_find_batch_size: False
log_on_each_node: True
logging_nan_inf_filter: True
include_num_input_tokens_seen: no
log_level: passive
log_level_replica: warning
disable_tqdm: False
project: huggingface
trackio_space_id: None
trackio_bucket_id: None
trackio_static_space_id: None
per_device_eval_batch_size: 32
prediction_loss_only: True
eval_on_start: False
eval_do_concat_batches: True
eval_use_gather_object: False
eval_accumulation_steps: None
include_for_metrics: []
batch_eval_metrics: False
save_only_model: False
save_on_each_node: False
enable_jit_checkpoint: False
push_to_hub: False
hub_private_repo: None
hub_model_id: None
hub_strategy: every_save
hub_always_push: False
hub_revision: None
load_best_model_at_end: True
ignore_data_skip: False
restore_callback_states_from_checkpoint: False
full_determinism: False
seed: 42
data_seed: None
use_cpu: False
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_pin_memory: True
dataloader_persistent_workers: False
dataloader_prefetch_factor: None
dataloader_multiprocessing_context: None
dataloader_in_order: True
remove_unused_columns: True
label_names: None
train_sampling_strategy: random
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
ddp_static_graph: None
ddp_backend: None
ddp_timeout: 1800
fsdp: None
fsdp_config: None
deepspeed: None
debug: []
skip_memory_metrics: True
do_predict: False
resume_from_checkpoint: None
local_rank: -1
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}
warmup_ratio: None
max_length: None

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss	NanoMSMARCO_maxsim_ndcg@10	NanoNQ_maxsim_ndcg@10	NanoFiQA2018_maxsim_ndcg@10	NanoBEIR_mean_maxsim_ndcg@10	NanoClimateFEVER_maxsim_ndcg@10	NanoDBPedia_maxsim_ndcg@10	NanoFEVER_maxsim_ndcg@10	NanoHotpotQA_maxsim_ndcg@10	NanoNFCorpus_maxsim_ndcg@10	NanoQuoraRetrieval_maxsim_ndcg@10	NanoSCIDOCS_maxsim_ndcg@10	NanoArguAna_maxsim_ndcg@10	NanoSciFact_maxsim_ndcg@10	NanoTouche2020_maxsim_ndcg@10	NanoClimateFEVER_pool_factor_1.5_maxsim_ndcg@10	NanoDBPedia_pool_factor_1.5_maxsim_ndcg@10	NanoFEVER_pool_factor_1.5_maxsim_ndcg@10	NanoFiQA2018_pool_factor_1.5_maxsim_ndcg@10	NanoHotpotQA_pool_factor_1.5_maxsim_ndcg@10	NanoMSMARCO_pool_factor_1.5_maxsim_ndcg@10	NanoNFCorpus_pool_factor_1.5_maxsim_ndcg@10	NanoNQ_pool_factor_1.5_maxsim_ndcg@10	NanoQuoraRetrieval_pool_factor_1.5_maxsim_ndcg@10	NanoSCIDOCS_pool_factor_1.5_maxsim_ndcg@10	NanoArguAna_pool_factor_1.5_maxsim_ndcg@10	NanoSciFact_pool_factor_1.5_maxsim_ndcg@10	NanoTouche2020_pool_factor_1.5_maxsim_ndcg@10	NanoBEIR_mean_pool_factor_1.5_maxsim_ndcg@10	NanoClimateFEVER_pool_factor_2_maxsim_ndcg@10	NanoDBPedia_pool_factor_2_maxsim_ndcg@10	NanoFEVER_pool_factor_2_maxsim_ndcg@10	NanoFiQA2018_pool_factor_2_maxsim_ndcg@10	NanoHotpotQA_pool_factor_2_maxsim_ndcg@10	NanoMSMARCO_pool_factor_2_maxsim_ndcg@10	NanoNFCorpus_pool_factor_2_maxsim_ndcg@10	NanoNQ_pool_factor_2_maxsim_ndcg@10	NanoQuoraRetrieval_pool_factor_2_maxsim_ndcg@10	NanoSCIDOCS_pool_factor_2_maxsim_ndcg@10	NanoArguAna_pool_factor_2_maxsim_ndcg@10	NanoSciFact_pool_factor_2_maxsim_ndcg@10	NanoTouche2020_pool_factor_2_maxsim_ndcg@10	NanoBEIR_mean_pool_factor_2_maxsim_ndcg@10	NanoClimateFEVER_pool_factor_3_maxsim_ndcg@10	NanoDBPedia_pool_factor_3_maxsim_ndcg@10	NanoFEVER_pool_factor_3_maxsim_ndcg@10	NanoFiQA2018_pool_factor_3_maxsim_ndcg@10	NanoHotpotQA_pool_factor_3_maxsim_ndcg@10	NanoMSMARCO_pool_factor_3_maxsim_ndcg@10	NanoNFCorpus_pool_factor_3_maxsim_ndcg@10	NanoNQ_pool_factor_3_maxsim_ndcg@10	NanoQuoraRetrieval_pool_factor_3_maxsim_ndcg@10	NanoSCIDOCS_pool_factor_3_maxsim_ndcg@10	NanoArguAna_pool_factor_3_maxsim_ndcg@10	NanoSciFact_pool_factor_3_maxsim_ndcg@10	NanoTouche2020_pool_factor_3_maxsim_ndcg@10	NanoBEIR_mean_pool_factor_3_maxsim_ndcg@10	NanoClimateFEVER_pool_factor_4_maxsim_ndcg@10	NanoDBPedia_pool_factor_4_maxsim_ndcg@10	NanoFEVER_pool_factor_4_maxsim_ndcg@10	NanoFiQA2018_pool_factor_4_maxsim_ndcg@10	NanoHotpotQA_pool_factor_4_maxsim_ndcg@10	NanoMSMARCO_pool_factor_4_maxsim_ndcg@10	NanoNFCorpus_pool_factor_4_maxsim_ndcg@10	NanoNQ_pool_factor_4_maxsim_ndcg@10	NanoQuoraRetrieval_pool_factor_4_maxsim_ndcg@10	NanoSCIDOCS_pool_factor_4_maxsim_ndcg@10	NanoArguAna_pool_factor_4_maxsim_ndcg@10	NanoSciFact_pool_factor_4_maxsim_ndcg@10	NanoTouche2020_pool_factor_4_maxsim_ndcg@10	NanoBEIR_mean_pool_factor_4_maxsim_ndcg@10
-1	-1	-	-	0.1282	0.1376	0.1222	0.1293	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0103	4	45.0892	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0207	8	43.0208	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0310	12	39.6614	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0413	16	34.9479	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0517	20	23.1162	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0620	24	16.2527	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0724	28	12.5760	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0827	32	10.3407	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0930	36	9.4709	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1008	39	-	6.7083	0.4070	0.4760	0.3427	0.4086	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1034	40	7.9055	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1137	44	7.4018	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1240	48	6.7647	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1344	52	6.1770	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1447	56	6.4575	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1550	60	6.0655	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1654	64	5.5661	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1757	68	5.1513	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1860	72	5.2597	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1964	76	4.7617	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2016	78	-	4.2965	0.4845	0.5005	0.3833	0.4561	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2067	80	5.1130	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2171	84	4.7112	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2274	88	4.3772	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2377	92	4.4892	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2481	96	4.8761	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2584	100	4.7895	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2687	104	4.5178	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2791	108	4.4951	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2894	112	4.3464	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2997	116	3.9389	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3023	117	-	3.6696	0.5011	0.5063	0.3948	0.4674	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3101	120	4.0816	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3204	124	3.9231	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3307	128	4.3137	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3411	132	3.8219	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3514	136	3.9318	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3618	140	3.9923	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3721	144	3.8811	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3824	148	3.7770	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3928	152	3.8564	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4031	156	3.8215	3.3162	0.5314	0.5140	0.3560	0.4672	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4134	160	3.5792	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4238	164	3.5880	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4341	168	3.5780	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4444	172	3.5824	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4548	176	3.9791	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4651	180	3.5915	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4755	184	3.6026	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4858	188	3.5513	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4961	192	3.3673	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5039	195	-	3.2542	0.5452	0.5521	0.3572	0.4848	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5065	196	3.6256	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5168	200	3.5818	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5271	204	3.5946	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5375	208	3.4593	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5478	212	3.2880	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5581	216	3.6941	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5685	220	3.8828	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5788	224	3.4232	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5891	228	3.2352	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5995	232	3.4714	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6047	234	-	3.1042	0.5014	0.5522	0.3589	0.4708	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6098	236	3.0802	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6202	240	3.1670	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6305	244	3.4719	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6408	248	3.3681	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6512	252	3.0772	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6615	256	3.0463	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6718	260	3.1527	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6822	264	3.2125	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6925	268	3.4101	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7028	272	3.4636	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7054	273	-	3.0435	0.5309	0.5512	0.3730	0.4850	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7132	276	3.3367	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7235	280	3.2590	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7339	284	3.4287	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7442	288	3.1293	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7545	292	3.4307	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7649	296	3.4242	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7752	300	3.0802	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7855	304	3.2275	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7959	308	3.3594	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8062	312	3.5244	2.9688	0.5250	0.5466	0.3724	0.4813	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8165	316	3.1145	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8269	320	3.3377	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8372	324	3.3492	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8475	328	3.2464	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8579	332	3.1393	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8682	336	3.1500	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8786	340	3.3753	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8889	344	2.9488	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8992	348	3.2033	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9070	351	-	2.9289	0.5360	0.5422	0.3800	0.4861	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9096	352	3.3311	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9199	356	3.0430	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9302	360	3.1809	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9406	364	2.9093	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9509	368	3.3547	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9612	372	3.0745	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9716	376	3.2533	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9819	380	3.2417	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9922	384	3.4003	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
1.0	387	-	2.9335	0.5373	0.5586	0.3746	0.4901	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
-1	-1	-	-	0.5373	0.5586	0.3746	0.5147	0.2397	0.5019	0.7899	0.6920	0.3052	0.9145	0.3207	0.3649	0.6479	0.4434	0.2291	0.4925	0.7948	0.3479	0.6762	0.5150	0.3013	0.5258	0.9012	0.3200	0.3529	0.6453	0.4223	0.5019	0.2487	0.5029	0.7905	0.3715	0.6590	0.5252	0.2950	0.5409	0.8932	0.3126	0.3543	0.6404	0.4361	0.5054	0.2387	0.5019	0.7908	0.3523	0.6443	0.5269	0.2943	0.5241	0.8293	0.3129	0.3644	0.6677	0.4248	0.4979	0.2391	0.4770	0.8043	0.3508	0.6216	0.5414	0.2881	0.4905	0.7381	0.3056	0.3464	0.6509	0.4318	0.4835

The bold row denotes the saved checkpoint.

Training Time

Training: 1.8 hours
Evaluation: 8.4 minutes
Total: 1.9 hours

Framework Versions

Python: 3.11.6
Sentence Transformers: 6.1.0.dev0
Transformers: 5.15.0
PyTorch: 2.10.0+cu128
Accelerate: 1.14.0
Datasets: 4.8.4
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultiVectorTokenPoolingLoss

@inproceedings{josef2026learntopool,
    title={Learn to Pool: Lightweight Fine-Tuning for Flexible Multi-Vector Compression},
    author={Stefan Josef},
    booktitle={1st Late Interaction Workshop (LIR) at ECIR 2026},
    year={2026},
}

CachedMultiVectorMultipleNegativesRankingLoss

@misc{gao2021scaling,
    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
    year={2021},
    eprint={2101.06983},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}