all-MiniLM-L6-v68-pair_score

This is a sentence-transformers model finetuned from KhaledReda/all-MiniLM-L6-v67-pair_score on the pairs_with_scores_v55 dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: KhaledReda/all-MiniLM-L6-v67-pair_score
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • pairs_with_scores_v55
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'manaeesh mat',
    'submersible pump deep water pumping discover the submersible pump available in a range of specifications tailored to meet your specific needs submersible pump 6m ideal for deep-water pumping tasks providing efficient water transfer up to 6 meters deep. submersible pump 2.8m perfect for medium-depth applications offering reliable performance for water transfer up to 2.8 meters deep. submersible pump 2.5m designed for versatile use capable of handling water transfer tasks up to 2.5 meters deep with ease. submersible pump 1.8m a versatile option for various pumping needs delivering efficient water transfer up to 1.8 meters deep. submersible pump 1.5m suitable for shallow-water pumping tasks providing reliable performa',
    'blackberry jam low carb jam diabetic jam friendly jam gluten free jam blackberry marmalade blackberry preserve blackberry - stevia - erythritol - gelatin nutritional facts per jar 103 kcal - 1.2 gr fat - 10.4 gr net carb - 12.7 gr fiber - 3.3 gr protein suitable for low carb - diabetic friendly - gluten free - 14 days in fridge.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.0196,  0.2491],
#         [ 0.0196,  1.0000, -0.0921],
#         [ 0.2491, -0.0921,  1.0000]])

Training Details

Training Dataset

pairs_with_scores_v55

• Dataset: pairs_with_scores_v55 at 73e55fb
• Size: 16,807,177 training samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 4 tokens mean: 7.42 tokens max: 21 tokens	min: 3 tokens mean: 56.69 tokens max: 256 tokens	min: 0.0 mean: 0.2 max: 1.0

• Samples:

  sentence1	sentence2	score
  microwave safe dish	car diffuser interior accessory	0.0
  vitacare turmeric curcumin	vaultix safe - vx680 2ed filing cabinet safe filing cupboard safe vaultix safe - vx 680 2 ed filing cupboard item code vx680 2ed 2 drawers . outside h82 w 51.5 d 68 cm in - side 2x h 30 w 37 d 52 cm . approx wt 150 10 kg.	0.0
  colorful swing	closeup - white now toothpaste 75 ml with free toothbrush - 1 pack closeup toothbrush closeup now closeup - now tooth polish with free toothbrush closeup tooth polish	0.0

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

Evaluation Dataset

pairs_with_scores_v55

• Dataset: pairs_with_scores_v55 at 73e55fb
• Size: 84,459 evaluation samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 3 tokens mean: 7.39 tokens max: 22 tokens	min: 4 tokens mean: 58.17 tokens max: 256 tokens	min: 0.0 mean: 0.21 max: 1.0

• Samples:

  sentence1	sentence2	score
  lade wide - leg washed jeans	sapphire 1.0 flared jeans pocket jeans women jeans sapphire flared jeans sapphire jeans retro inspired with contemporary edge these jeans are a sure stunner with every wear. take note of the butterfly wings pockets wink	1.0
  maybelline super stay vinyl ink	plumping liquid no. 884 volume natural look polyisobutene paraffin oil beeswax longlasting perfumed plumping liquid . liquid for a full and attractive look. fuller gorgeous lips. immediate volume. natural look. contains polyisobutene. paraffin oil paraffinum liquidum . beeswax. isopropyl. palmitate. methylparaben. propylparaben. colors. perfume.	1.0
  nigiri prawn rice	jacks dutch gouda goat roquefort blue vein cheese bold flavor cheese dutch cheese jacks dutch cheese goat cheese gouda cheese jacks cheese roquefort cheese chevre cheese jacks dutch gouda chevre roquefort jacks gbnah roquefort gbnah this artisanal cheese boasts a distinctive blue vein pattern delivering a bold and tangy flavor profile.	0.0

• Loss: CoSENTLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• fp16: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Click to expand

Framework Versions

• Python: 3.12.3
• Sentence Transformers: 5.1.0
• Transformers: 4.55.4
• PyTorch: 2.6.0+cu124
• Accelerate: 1.10.1
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CoSENTLoss

@online{kexuefm-8847,
    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
    author={Su Jianlin},
    year={2022},
    month={Jan},
    url={https://kexue.fm/archives/8847},
}