BGE-Small Fine-Tuned on USCode-QueryPairs

This is a fine-tuned version of the BGE Small embedding model, trained on the USCode-QueryPairs dataset, a subset of the USLawQA corpus. The model is optimized for generating embeddings for legal text, achieving 75% accuracy on the test set.

Overview

• Base Model: BGE Small
• Dataset: USCode-QueryPairs
• Training Details:
  • Hardware: Google Colab (T4 GPU)
  • Training Time: 2 hours
• Accuracy: 75% on the test set from USLawQA

Applications

This model is ideal for:

• Legal Text Retrieval: Efficient semantic search across legal documents.
• Question Answering: Answering legal queries based on context from the US Code.
• Embeddings Generation: Generating high-quality embeddings for downstream legal NLP tasks.

Usage

The model can be used with model.encode for generating embeddings. Below is an example usage snippet:

# Load model directly
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("ArchitRastogi/BGE-Small-LegalEmbeddings-USCode")
model = AutoModel.from_pretrained("ArchitRastogi/BGE-Small-LegalEmbeddings-USCode")
text = "Duties of the president"
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)
#Printing the Embeddings
print(outputs)

Evaluation

The model was evaluated on the test set of USLawQA and achieved the following metrics:

• Accuracy: 75%
• Task: Semantic similarity and legal question answering.

Related Resources

• USCode-QueryPairs Dataset
• USLawQA Corpus

📧 Contact

For any inquiries, suggestions, or feedback, feel free to reach out:

Archit Rastogi
📧 architrastogi20@gmail.com

---

📜 License

This dataset is distributed under the Apache 2.0 License. Please ensure compliance with applicable copyright laws when using this dataset.