For more details please refer to our research paper and GitHub repo

BGE-Finetuned (paper, code)

In this project, we fine-tuned the BGE-M3 dense embeddings model on three low-resource Nigerian languages (Yoruba, Hausa, and Igbo). We also enabled cross-lingual retrieval by finetuning the model on English queries mapped to each of the three languages.

Specs

• Model

• Data

Usage

import torch
from FlagEmbedding import BGEM3FlagModel

device = "cpu"
if torch.cuda.is_available():
    device = "cuda"
elif torch.backends.mps.is_available():
    device = "mps"

half_precision = False # or True if you really want
# Load the abdulmatinomotoso/bge-finetuned model
model = BGEM3FlagModel('abdulmatinomotoso/bge-finetuned', use_fp16=half_precision, devices=[device])

# Example text to generate embeddings for
documents = [
    "Eyi jẹ́ gbolohun àpẹẹrẹ",
    "Eyi kì í ṣe gbolohun àpẹẹrẹ",

    "Nke a bụ nkebisiokwu atụ",
    "Nke a abụghị nkebisiokwu atụ",

    "Wannan jimla ce ta misali",
    "Wannan ba jimla ce ta misali ba"
]

query = "Where is the example?"

# Generate embeddings
sparse_embeddings = False # The sparse embeddings are not useful as is, they will require some work to make the align with the entire model.
multivec_embeddings = False # The multivector embeddings are still useful despite training only for dense, but this sample uses only dense embeddings
dense_embeddings = True

doc_embeddings = model.encode(documents, return_sparse=sparse_embeddings, return_dense=dense_embeddings, return_colbert_vecs=multivec_embeddings)["dense_vecs"]

query_embeddings = model.encode([query], return_sparse=sparse_embeddings, return_dense=dense_embeddings, return_colbert_vecs=multivec_embeddings)["dense_vecs"]

similarity_scores = query_embeddings @ doc_embeddings.T
print(similarity_scores)
# array([[
#     0.4360781, # Higher similarity than the opposite text
#     0.40966046, 
    
#     0.4775736, # Higher similarity than the opposite text
#     0.44197953, 
    
#     0.4514127, # Higher similarity than the opposite text
#     0.41557986
# ]], dtype=float32)

Acknowledgement

Thanks to the authors of open-sourced datasets, including Miracl, and Wura. Thanks to the open-sourced libraries like Trafilatura, FlagEmbedding.

Citation

If you find this repository useful, please consider giving a star :star: and citation

@inproceedings{omotoso-etal-2025-improving,
  title     = {Improving BGE-M3 Multilingual Dense Embeddings for Nigerian Low Resource Languages},
  author    = {Omotoso, Abdulmatin and Shopeju, Habeeb and Joshua, Adejumobi Monjolaoluwa and Oni, Shiloh},
  booktitle = {Proceedings of the 9th Widening NLP Workshop},
  year      = {2025},
  month     = nov,
  address   = {Suzhou, China},
  publisher = {Association for Computational Linguistics},
  pages     = {224--229},
  doi       = {10.18653/v1/2025.winlp-main.33},
  url       = {https://aclanthology.org/2025.winlp-main.33/}
}